Knowdee at BLP -2023 Task 2: Improving B angla Sentiment Analysis Using Ensembled Models with Pseudo-Labeling (2023)

Abstract: This paper outlines our submission to the Sentiment Analysis Shared Task at the Bangla Language Processing (BLP) Workshop at EMNLP2023 (Hasan et al., 2023a). The objective of this task is to detect sentiment in each text by classifying it as Positive, Negative, or Neutral. This shared task is based on the MUltiplatform BAngla SEntiment (MUBASE) (Hasan et al., 2023b) and SentNob (Islam et al., 2021) dataset, which consists of public comments from various social media platforms. Our proposed method for this task is based on the pre-trained Bangla language model BanglaBERT (Bhattacharjee et al., 2022). We trained an ensemble of BanglaBERT on the original dataset and used it to generate pseudo-labels for data augmentation. This expanded dataset was then used to train our final models. During the evaluation phase, 30 teams submitted their systems, and our system achieved the second highest performance with F1 score of 0.7267. The source code of the proposed approach is available at https://github.com/KnowdeeAI/blp_task2_knowdee.git.

## introduction while english dominates as the most resourcerich language in the natural language processing (nlp) community, bangla which ranked as the 6th most spoken language still faces resource scarcity. despite three decades of bnlp research, progress has lagged mainly due to scarce resources and associated challenges @xcite . the objective of the sentiment analysis shared task is to detect sentiment in each text by classifying it as positive, negative, or neutral. this task utilizes a combined dataset of multiplatform bangla sentiment (mubase) @xcite and @xcite . mubase contains manually annotated social media posts from twitter and facebook labeled with sentiment polarity. sentnob consists of social media comments from multiple platforms related to news and videos covering 13 different domains @xcite . bangla is a language with rich morphology, many dialects, and unique linguistic nuances. @xcite . additionally, the dataset used consists of noisy social media comments with a mix of dialects and grammatical errors @xcite . the combination of bangla's inherent linguistic challenges and the informal, non-standard nature of the dataset creates difficulties for sentiment analysis. in this work, we present our solution and experimental attempts at the sentiment analysis shared task in section 2, our main approach involves an ensembling technique with pseudo-labeling to maximize performance given the limited training data. results and analysis are followed in section 3. finally, section 4 concludes with a summary of results and an outlook on future directions to advance low-resource natural language processing tasks for bangla and other languages. ## system description we discuss our proposed solution for the shared task from section 2.1 to section n three steps: 1) finetuning an ensemble of models on the provided supervised training data, 2) using the ensemble models from step 1 to generate pseudo-labels for unlabeled data, 3) training a new ensemble on the combination of the original training data and pseudo-labeled dataset, to make final predictions. additionally, we discuss other pre-trained models we experimented using the proposed solution and another attempted solution in section 2.4. the experiments result is discussed in section 3.2. ## experiments and results this section presents the official results of our submitted solution for the sentiment analysis shared task. additionally, we conducted post-evaluation experiments using the gold standard labels to compare the performance of our submitted system against alternative approaches on the test set. ## conclusion and future work in this work, we presented our approach and results for the sentiment analysis shared task. our proposed solution using banglabert_large achieved strong performance, ranking 2nd out of 30 submitted systems with an f1-micro score of 0.7267. through post-competition analysis, we found that larger transformer models designed specifically for indian languages, such as banglabert and muril, lead to better performance on this multi-class senti-ment analysis task. for low-resource languages like bangla, pretrained models tailored to the specific language are crucial, as our results demonstrated the superior performance of bangla-focused models over multilingual models. however, when languages have limited resources, starting with multilingual models from related language families can provide an initial boost, as evidenced by the strong test set results of muril-large-cased pretrained on indian languages. as resources grow, continued pre-training of language-specific models on larger and more diverse corpora for that language can further improve adaptation. additionally, leveraging semisupervised approaches and generative data augmentation techniques to expand limited labeled datasets will become more viable. techniques like consistency training, backtranslation, and synthetic data generation can help in low-resource scenarios but require a certain data baseline to be effective.