Transformer-based B engali Textual Emotion Recognition (2023)

Abstract: Emotion recognition for high-resource languages has progressed significantly. However, resource-constrained languages such as Bengali have not advanced notably due to the lack of large benchmark datasets. Besides this, the need for more Bengali language processing tools makes the emotion recognition task more challenging and complicated. Therefore, we developed the largest dataset in this paper, consisting of almost 12k Bengali texts with six basic emotions. Then, we conducted experiments on our dataset to establish the baseline performance applying machine learning, deep learning, and transformer-based models as emotion classifiers. The experimental results demonstrate that the models achieved promising performance in Bengali emotion recognition.

## introduction since learning emotions through automated systems was a decades-long objective @xcite , recognizing textual emotion is crucial for assessing the enormous volumes of user data on various virtual platforms. the generated texts contain different user emotions expressed in their own manner, making the emotion classification task more challenging @xcite . however, @xcite defined six primary emotions (joy, sadness, surprise, disgust, anger and fear) can be extracted from texts with scientific techniques and tools @xcite and recognizing them has major usages for online businesses and system enhancements such as removing unwanted content @xcite ) from the system. the identification of emotions in highresource languages such as english, chinese, and others has advanced considerably @xcite @xcite due to the availability of advanced computational processes and tools. however, emotion detection has not progressed significantly in resourceconstrained languages such as bengali due to the lack of large benchmark datasets and language processing tools @xcite . there has been some research on emotion identification using machine learning (ml) and deep learning (dl) approaches. tripto and ali (2018) suggested an lstm-based multi-label emotion recognition system for youtube comments in bengali and english. another study by @xcite identified joyful, sad, and angry emotions in bengali texts. they applied the term frequency-inverse document frequency (tf-idf) and multinomial naive bayes for classification. ruposh and hoque (2019) used 1.2k texts to develop a framework for recognizing six raw emotions. they have proposed a method using bag-of-words (bow) with support vector machine (svm). @xcite presented a method for identifying happiness, anger, sadness, and suspense in bengali short stories using tf-idf with logistic regression (lr). @xcite has proposed a bengali emotion recognition dataset of 6.3k texts containing @xcite defined basic six emotions (anger, fear, disgust, sadness, joy, and surprise). authors have applied transformer-based models m-bert, bangla-bert @xcite and xlm-r to accomplish the task. these researchers utilized a small amount of data due to the scarcity of large datasets. to address the lack of a large benchmark dataset, we have constructed a benchmark bengali emotions dataset comprised of almost 12k (11927) texts of six distinct emotions (joy, sadness, surprise, disgust, anger, and fear). we also conducted experiments to determine the performance of ml (logistic regression (lr), random forest (rf), naive bayes (nb), support vector machine (svm) and ensemble method), dl (lstm, bil-stm, cnn-bilstm) and transformer-based (bangla-bert, mbert, electra bangla-bert, xlm-r) methods on our dataset and to set the baseline of this dataset. as baseline methods, lr, svm, and bangla-bert showed more promising performance on the developed dataset than other approaches. the contributions of this study are as follows: the rest of the paper is organized as follows: section 2 details some related works of bengali emotions classification. then, we describe our proposed method for bengali emotion classification in section 3. the experimental results and the details of the developed dataset are presented in section 4. finally, section 5 discusses the concluding statements of this paper. ## related works researchers have proposed different approaches to emotion identification tasks using emotional texts for various languages, including english, chinese, and arabic @xcite . hybrid and learning-based approaches that utilize traditional text representation with distributed word representation outperformed some existing approaches on benchmark corpora @xcite . graph convolution network architecture also showed good performance @xcite . recently, the transformer-based model (i.e., bert) has gained popularity among researchers in analyzing emotional texts. a new method based on electra and a hybrid neural network can avoid the inconsistency of the mask training and fine-tuning process of the traditional pre-training model proposed by @xcite . @xcite proposed a novel hierarchical lstms for contextual emotion detection (hrlce) model. they combined the results generated by bert and hrcle to achieve the overall performance. although emotion identification tasks on high-resource languages have attracted many researchers for further investigation, lowresource languages like bengali are still preliminary in research progress. few researchers proposed different methods employing ml and dl approaches. @xcite proposed k-nearest neighbour-based method on indonesian texts and showed 58% accuracy for the six-label classification and 79% accuracy for the two-label classification. @xcite identified multi-class emotions from bangla texts using multinomial naïve bayes (nb) classifier along with various features such as stemmer, parts-of-speech (pos) tagger, n-grams, tf-idf. they used three emotion classes (happy, sad, and angry) and reported an overall accuracy of 78.6%. dl models classified bangla sentences with three classes (positive, negative, neutral) and five (strongly positive, positive, neutral, negative, strongly negative) sentiment labels with 65.97% and 54.24% accuracy, respectively @xcite . @xcite presented an emotion recognition technique by developing a corpus consisting of 1200 emotive words that are used to train the svm classifier with 73% accuracy which is higher than the naive based approach (60%). however, our proposed dataset comprises almost 12k bangali emotional texts. ## methodology this section discusses the strategies we explored for developing baseline models for identifying bengali emotions on our developed dataset. we collected emotional texts from different community websites like facebook, youtube, and news portal comments. then, we applied different ml, dl, and transformerbased models to predict the type of emotional text. ## experiments and results this section details the dataset collection and annotation techniques along with the experimental results and analysis. ## conclusion this paper investigates the performance of different ml, dl, and transformer-based methods to classify the emotion of resourceconstrained bengali language. due to the scarcity of a large dataset, bengali emotion classification research has not achieved remarkable progress compared to other highresourced languages. therefore, we developed the largest bengali emotion recognition dataset comprising almost 12k texts containing six basic human emotions. then, we employed different ml, dl, and transformerbased approaches to set the baseline performances for the developed dataset. the experimental results showed that the ml and transformer-based methods better classify emotions than dl models. moreover, the results also indicate the challenges in identifying emotions in low-resource languages like bengali. therefore, in the future, we would like to increase the size of this dataset and develop a large corpus for different resourceconstrained languages with more emotions for low-resourced languages.