CUET - NLP @ T amil NLP - ACL 2022: Multi-Class Textual Emotion Detection from Social Media using Transformer (2022)

Abstract: Recently, emotion analysis has gained increased attention by NLP researchers due to its various applications in opinion mining, e-commerce, comprehensive search, healthcare, personalized recommendations and online education. Developing an intelligent emotion analysis model is challenging in resource-constrained languages like Tamil. Therefore a shared task is organized to identify the underlying emotion of a given comment expressed in the Tamil language. The paper presents our approach to classifying the textual emotion in Tamil into 11 classes: ambiguous, anger, anticipation, disgust, fear, joy, love, neutral, sadness, surprise and trust. We investigated various machine learning (LR, DT, MNB, SVM), deep learning (CNN, LSTM, BiLSTM) and transformer-based models (Multilingual-BERT, XLM-R). Results reveal that the XLM-R model outdoes all other models by acquiring the highest macro f_1 -score (0.33).

## introduction textual emotion analysis is the automatic process of specifying a text into an emotion class from predefined connotations @xcite . with the unprecedented growth of the internet, online and social media platforms significantly influence people's lives and interactions. people share opinions, expressions, information, feelings, emotions, ideas and concerns online @xcite . people seek emotional support from their relatives, friends, or even virtual platforms when they go through challenging or adverse times @xcite @xcite . textual emotion analysis (tea) has been proven helpful in various applications, for example, consumer feedback on services and products @xcite . the positive and negative customer experiences help to assess the demand for products and services @xcite . however, one cannot fully express his/her attitude only through positive and negative sentiments. for example -i threw my iphone in the water, and now it is not working, so i feel awful (sadness) vs what a pain my new iphone is not working (anger). both texts express negative sentiment, but the first is sadness, and the latter is considered anger. thus, emotion analysis is very crucial to understand the actual state of mind @xcite . in recent years, plenty of research has been conducted to analyze textual emotion. however, low-resource languages (i.e. tamil and bengali) remained out of focus, and very few research activities have been conducted to date. this deficiency occurs due to the scarcity of resources, limited corpora and unavailability of text processing tools @xcite @xcite @xcite . this shared task paper aims to mitigate this gap by presenting computational models for emotion analysis in tamil. tamil is the predominant language of the majority of people living in tamil nadu, puducherry (in india), and the northern and eastern regions of sri lanka @xcite @xcite @xcite . the language is spoken by tiny minority communities in various indian states such as karnataka, andhra pradesh, kerala, maharashtra, and in specific places of sri lanka such as colombo and the hill country. tamil or varieties of it were widely employed as the main language of governance, literature, and general usage in the state of kerala until the 15th century ad @xcite @xcite . tamil was also commonly employed in inscriptions unearthed in the southern andhra pradesh regions of chittoor and nellore until the 12th century ad. tamil was employed for inscriptions in southern karnataka regions such as kolar, mysore, mandya, and bangalore from the 10th to 14th century @xcite . the significant contribution of this work illustrates in the following: in the past few years, emotion analysis research has attracted researchers from diverse domains such as computer science, psychology and healthcare. chaffar and inkpen (2011) developed a model to recognize six basic emotions from the affective text on alm's dataset (1250 texts). they employed several ml techniques where support vector machine (svm) achieved the highest performance with bag of words (bow) features. huang et al. (2019) proposed a contextual model to detect emotion. they combined two lstm layers hierarchically and formed an ensemble with the bert model, which achieved 77% accuracy. vijay et al. (2018) developed a model with svm and rbf kernel to identify the fear, disgust and surprise emotions from 2866 hindi-english codemixed tweets. wadhawan and aggarwal (2021) experimented with several dl (cnn, lstm, bil-stm) and transformer-based models for recognizing emotions from 149088 hindi-english code mixed tweets. the transformer-based bert model outperformed all other techniques and obtained an accuracy of 71.43%. iqbal et al. (2022) presented a bengali emotion corpus (bemoc) containing 7000 texts with six basic emotion categories: joy, anger, sad, fear, surprise, disgust. das et al. (2021) performed an investigation of various ml, dnn, and transformer-based techniques on bemod dataset containing 6523 texts. their results showed that xlm-r outdoes others providing an f 1 -score of 69.61%. in a similar work, parvin et al. (2022) implemented various dl techniques (cnn, gru, bilstm) with different ensemble combinations to recognize six emotions from a corpus containing 9000 bengali texts. the ensemble of cnn and bilstm outperformed other models by achieving f 1 -score of 62.46%. ## task and dataset descriptions the emotion analysis shared task in tamil comprises two tasks. we have participated in task-a, where multi-class categorization of textual emotion is performed. the organizers foot_0 @xcite provided the annotated dataset having 11 types of emotions: ambiguous, anger, anticipation, disgust, fear, joy, love, neutral, sadness, surprise and trust. the dataset consists of training, validation and test sets containing 14208, 3552 and 4440 texts. to get better insights, we further analyzed the training set. ## methodology this work employed four ml, three dl and two transformer-based approaches to identify the underlying emotions of social media comments in ## results and analysis however, the transformer model, xlm-r, outperformed all the models by achieving the highest accuracy (0.47), precision (0.36), recall (0.33) and macro f 1 -score (0.33). ## conclusion this paper investigated four ml, three dl and two transformer-based models to classify emotion from tamil texts. among all models, the xlm-r obtained the highest macro f 1 -score of 0.33. since this work did not use any pre-trained embedding, it might adversely affect the performance of the dl model. thus, we opt to experiment with pretrained word embedding in the future. moreover, we plan to explore other advanced transformerbased models (i.e., indic-bert, muril) and ensemble approaches to address the emotion analysis task. since the dataset is imbalanced, it will be interesting to investigate the impact of resampling on the models in the future.