Findings of the Shared Task on Emotion Analysis in T amil (2022)

Abstract: This paper presents the overview of the shared task on emotional analysis in Tamil. The result of the shared task is presented at the workshop. This paper presents the dataset used in the shared task, task description, and the methodology used by the participants and the evaluation results of the submission. This task is organized as two Tasks. Task A is carried with 11 emotions annotated data for social media comments in Tamil and Task B is organized with 31 fine-grained emotion annotated data for social media comments in Tamil. For conducting experiments, training and development datasets were provided to the participants and results are evaluated for the unseen data. Totally we have received around 24 submissions from 13 teams. For evaluating the models, Precision, Recall, micro average metrics are used.

## introduction emotional analysis is the process of mining emotions in texts for categorization, which is used in a variety of natural language applications such as e-commerce review analysis, public opinion analysis, comprehensive search, customized suggestion, health-care, and online teaching. emotions are mental states that are frequently represented through behavior or words. emotional analysis assists in analyzing the text and extracting the emotions portrayed in the text. there are a variety of monolingual datasets available for dravidian languages, which may be used for a variety of research purposes @xcite @xcite . there have, however, been few attempts to create codemixed datasets for tamil, kannada, and malayalam @xcite . we will assist tamil in overcoming this resource barrier by producing this dataset, which will provide tamil with cost-effective and quick natural language processing support in emotional analysis. we gathered comments on several tamil movie trailers and teasers from youtube to develop resources for a tamil scenario. data on social media platforms such as twitter, facebook, and youtube is rapidly changing and may have a significant impact on an individual's or community's perception or reputation @xcite @xcite . this highlights the need of automation in emotional analysis. because of its numerous content offerings, such as movies, trailers, music, tutorials, product reviews, and so on, youtube is a popular social media network in the indian subcontinent @xcite . youtube allows users to create content as well as participate and interact with it through activities such as like and commenting. as a result, more user-generated content is available in underdeveloped languages @xcite @xcite . in this paper the results of the shared task on emotional analysis in tamil are presented and held at dravidianlangtech. this task consists of two sub tasks, namely task a and task b. the first task is focused on analysing the emotion in social media tamil comments with 11 emotions. the second task is focused on analysing 31 fine-grained emotions in social media tamil comments. the task is focused on analysing emotions on tamil language which is a low resource language that belongs to dravidian family. all the data including training data, testing data, development data along with results are made publicly available. we strongly believe that the curated dataset will contribute more in advancing the research in the field of emotional analysis field of tamil. ## related work as the availability of text data is more today, there are many good works carried out in the field of natural language processing (nlp). identifying the emotions in a given text data helps in various applications, including online teaching, recommendation systems, public opinion analysis. even though many researchers contribute in emotion analysis domain, very few works have been made in dravidian languages like tamil. in recent days, deep learning based techniques yield good results in this domain. with social media content, emotion detection from code mixed hindi and english language (sasidhar et al., 2020) is performed. they created an emotion annotated corpus using 12000 code-mixed text sentences from different social media sites as the data availability in their field of interest is less. it is reported that the cnn-bilstm model provides good accuracy. @xcite performed emotion analysis using the proposed cnn_text_word2vec model in chinese microblog. all the words in the microblog are trained using word2vec neural network model. the trained feature vectors are fed as input feature for attention-based convolution neural network model which classifies the text emotion into positive and negative. sina weibo microblog is used for experimental analysis in which 40000 training and 40000 test microblog samples were considered. a combination of cognitive linguistic and deep learning features are used for performing emotion analysis on persian text @xcite . these models help to identify emotions including anger, happiness, sadness, surprise and fear. for performance measurement, a self-labeled 23,000 persian documents are taken. they used word2vec embedding technique for vector conversion. the deep learning based architectures @xcite such as artificial neural network, convolutional neural network, and long short-term memory based recurrent neural network are examined for turkish tweets emotional prediction (s.anbukkarasi and s.varadhaganapathy, 2020). to curate a dataset, the lexicon-based approach is proposed for automatic annotation of turkish tweet text and compared the deep learning based architectures that outperformed the traditional machine learning techniques for emotion recognition in turkish. hybrid model is proposed to identify the emotions in arabic text @xcite . human-engineered feature-based model and deep feature-based model are investigated in which human-engineered feature-based model is used to select the features based on various kinds of the text like lexical, stylistics, and semantic. deep feature based model contains stacked neural network that reinserts the embedding layer several times to delay the learning process. from the study, it is clear that there is a research gap in emotional analysis field for tamil language with advanced techniques. ## evaluation in this section we describe about the evaluation method that were used for the both tasks a and b. we primarily evaluated the submissions using major classification metrics such as macro averaged and weighted average precision, recall and f1-score. for ranking the teams, we primarily used macro averaged f1 score as it finds f1 score for each label and find their unweighted mean. ## results and discussion fine-grained emotion detection is a hard problem, it is even harder for low resource languages such as tamil, in this shared task teams have submitted results for both tasks a and b. figures 1 and 2 depicts the pictorial representation of scores of each team for both task a and task b. the results are discussed in ## conclusion and future work two new datasets with fine-grained emotions for classifying the emotions in tamil language is created for this shared task. in this paper, various approaches used for analysing emotions from tamil comments and their results are analyzed. the task has been carried out as two sub tasks. in the task a, 8 teams have participated and submitted their results. for task b, 7 teams submitted their results. the xlm-r model produces the highest f1-score for task a and the combination of techniques including linguistic features, pos tags, contextual embeddings yields remarkable results for task b. as a future work, we planned to increase the size of the dataset and include more fine grained emotional labels to increase the performance of the system.