MUNLP @ D ravidian L ang T ech2023: Learning Approaches for Sentiment Analysis in Code-mixed T amil and T ulu Text (2023)

Abstract: Sentiment Analysis (SA) examines the subjective content of a statement, such as opinions, assessments, feelings, or attitudes towards a subject, person, or a thing. Though several models are developed for SA in high-resource languages like English, Spanish, German, etc., uder-resourced languages like Dravidian languages are less explored. To address the challenges of SA in low resource Dravidian languages, in this paper, we team MUNLP describe the models submitted to “Sentiment Analysis in Tamil and Tulu- DravidianLangTech” shared task at Recent Advances in Natural Language Processing (RANLP)-2023. n-gramsSA, EmbeddingsSA and BERTSA are the models proposed for SA shared task. Among all the models, BERTSA exhibited a maximum macro F1 score of 0.26 for code-mixed Tamil texts securing 2nd place in the shared task. EmbeddingsSA exhibited maximum macro F1 score of 0.53 securing 2nd place for Tulu code-mixed texts.

## introduction the dynamic nature of social media platforms like twitter, facebook, and youtube, characterized by rapidly evolving user-generated content, un-derscores the importance of automated sa, opinion mining, hate speech detection and offensive language detection @xcite ). analyzing the sentiments of opinions, reviews, comments, etc., about the photos, videos, songs, movies or anything, on social media platforms such as youtube and facebook, can offer valuable insights to the organizations and individuals to make informed decisions about the content @xcite @xcite @xcite . by analyzing the opinions of the users', content providers can also tailor their strategies to better align the content with the preferences and expectations of users, fostering a stronger connection with the users with the intention of potentially enhancing their reputation. such sentiment-driven insights enable effective engagement, enabling them to address concerns, capitalize on positive feedback, and adapt their approaches for the improved outcomes @xcite @xcite . sa is the process of computationally identifying and categorizing opinions, emotions, and attitudes expressed in written or spoken language. it has emerged as a crucial field of study due to the significant marks it leaves on the online users. sa is witnessing the growing adoption on social media platforms like youtube, where it is leveraged as a recommender system due to its significant impact on viewers. @xcite .. technological limitations often lead users to express their sentiments and opinions in their native languages using the roman script, along with the inclusion of english words. this practice arises from the convenience of keying in roman letters compared to the more complex key combinations required for native language scripts, particularly in the case of indian languages @xcite . this results in code-mixed text, where more than one languages are combined at sentence, word, or sub-word level within the same text. with the rise of social media platforms and user generated content like sentiments and emotions, code-mixing texts have become increasingly prevalent as users find it easier to communicate their thoughts mixing the words of different languages they are very much familiar. the complexity of sa is significantly increased in code-mixed texts as they combine words from different languages at various linguistic-levels @xcite ). analyzing sentiments in such texts require the language models and algorithms that are capable of effectively handling code-mixed content, accurately identifying sentiment-bearing units, and deciphering the sentiments expressed in different languages. overcoming these challenges is crucial for obtaining accurate and meaningful insights for codemixed sa. indian languages in general and dravidian languages in particular, are under-resourced languages and code-mixing adds further dimension, mainly due to the problems with collecting and annotating code-mixed data for various applications. "sentiment analysis in tamil and tulu" is a shared task in dravidianlangtech at ranlp 2023 with the aim of promoting sa of code-mixed texts in tamil and tulu @xcite . we, team mucs, submitted three distinct models: i) n-gramssa -an n-grams based model in which term frequency-inverse document frequency (tf-idf) of word n-grams and char wb in the range (1, 3) are used to train linearsvc, ii) embeddingssa -a linear svc model trained using concatenation of fasttext and byte pair word embeddings, and iii) bertsa -a model constructed by fine-tuning tamil sentiment bert (only for tamil language), to address the challenges provided by the shared task. the rest of paper is organized as follows: while section 2 describes the recent literature on codemixed text processing and sa, section 3 focuses on the description of the models submitted to the shared task followed by the experiments and results in section 4. conclusion and future works are included in section 5. from the above literature, it is clear that the researchers explored several models to perform sa in both high-resource and low-resource languages. however, for most of the low-resource languages, performances of the models are still less indicating the scope for developing models for sa in lowresource languages. further, the code-mixed nature of the social media comments in low-resource languages intensifies the sa task. ## methodology three distinct models: n-gramssa, embed-dingssa, and bertsa, are proposed for sa of code-mixed tamil and tulu texts. the steps involved in building the proposed models are given below: ## experimental results the dataset provided by the shared task organizers includes code-mixed tamil @xcite and tulu @xcite texts and the distribution of the labels across the train and development sets for these two languages are shown in the results illustrate that, both n-gramssa and embeddingssa models exhibited similar performance with the same macro f1 score of 0.18. further, bertsa model outperformed the other models with a macro f1 score of 0.26 securing 2 nd rank for code-mixed tamil text. for tulu, among n-gramssa and embeddingssa models, embed-dingssa model outperformed the other with macro 8 https://huggingface.co/vishnun/bert-base-cased-tamilmix-sentiment f1 score of 0.53 on test set securing 2 nd rank in the shared task. however, the macro f1 score obtained by the proposed models are below average. the macro f1 scores are very low due to the imbalance in the train set. though the hyperparameter 'class weight' set to 'balanced' resolves the data imbalance issue to some extent, the extreme data imbalance in the train set leads to overfitting. the misclassified comments in tamil and tulu test sets along with the english translations, actual labels, model predictions (obtained from bertsa and embeddingssa models evaluated for tamil and tulu test sets respectively), and probable reasons for misclassification are shown in the comparison of macro f1 scores of all the participating teams for sa in both code-mixed tamil and tulu text is shown in figure 2 . ## conclusion this paper describes the models submitted by our team -munlp, to the shared task "sentiment analysis in tamil and tulu" in dravidianlangtech at ranlp 2023, for sa in code-mixed tamil and tulu texts. three distinct models: i) n-gramssaan n-grams based model trained with n-grams of words and char wb both in the range (1, 3) are used to train linearsvc, ii) embeddingssa -a model built using combination of fasttext and byte pair embeddings to train linearsvc, and iii) bertsa -a model constructed by fine-tuning tamil sentiment bert (only for tamil language), to address the challenges of the shared task. among the proposed models, bertsa and embeddingssa models obtained macro f1 scores of 0.26 and 0.53 for code-mixed tamil and tulu datasets respectively both securing 2 nd rank. the results indicate that the macro f1 scores are low. suitable oversampling or text augmentation techniques will be explored to improve macro f1