MUCSD @ D ravidian L ang T ech2023: Predicting Sentiment in Social Media Text using Machine Learning Techniques (2023)

Abstract: User-generated social media texts are a blend of resource-rich languages like English and low-resource Dravidian languages like Tamil, Kannada, Tulu, etc. These texts referred to as code-mixing texts are enriching social media since they are written in two or more languages using either a common language script or various language scripts. However, due to the complex nature of the code-mixed text, in this paper, we - team MUCSD, describe a Machine learning (ML) models submitted to “Sentiment Analysis in Tamil and Tulu” shared task at DravidianLangTech@RANLP 2023. The proposed methodology makes use of ML models such as Linear Support Vector Classifier (LinearSVC), LR, and ensemble model (LR, DT, and SVM) to perform SA in Tamil and Tulu languages. The proposed LinearSVC model’s predictions submitted to the shared tasks, obtained 8th and 9th rank for Tamil-English and Tulu-English respectively.

## introduction social media usage is increasing continuously due to the ease of use to share reviews even for those who are not familiar about the technologies @xcite @xcite @xcite @xcite @xcite . this has led to the increased usergenerated text of opinions or reviews, comments, and posts on social media like twitter, facebook, youtube, etc @xcite . understanding the users' comments on topics/events in social media allows for more informed decision-making. it helps the content creators like youtubers to evaluate the emotional impact of their videos on viewers @xcite . for example, while positive comments about a video indicate satisfaction, negative comments indicate the need for improvement @xcite . sa is the task of analysing the reviews, opinions or comments to identify their polarity. this analysis provides insights into public opinions, ideas, and statements, which can be valuable for bloggers, researchers, and even individuals @xcite @xcite . social media posts/comments such as youtube comments, facebook posts, and tweets, often contain slang, misspellings, contractions, etc. which may impair the ability of the learning models to recognize patterns in the social media content and produce precise predictions. as there is no restriction on the use of language in social media, users often use more than one language to write comments on social media platforms. they may even use more than one script to key in the comments leading to code-mixed data. majority of the sa works focus on highresource languages like english and spanish giving less importance for low-resource languages and code-mixed low-resource languages @xcite . to promote the sa research work on low-resource languages, in this paper, weteam mucsd, describe the ml models submitted to the shared task on "sentiment analysis in tamil and tulu" at dravidianlangtech@ranlp 2023 1 @xcite . the aim of the shared task is to classify the comments in code-mixed tamil and tulu languages, into one of the four categories: positive, neutral/unknown state, mixed feelings, and negative. the proposed methodology makes use of linearsvc, lr, and ensemble of classifiers (lr, dt, and svm) with hard voting, trained with tf-idf of word unigrams to predict the sentiment of the given text. the rest of the paper is structured as follows: the related work is briefly described in section 2. section 3 describes the methodology, while section 4 discusses experiments with the results. the study concludes with future work in section 5. ## related work researchers have developed several ml approaches to handle monolingual and code-mixed texts for sa. a brief description of few of the relevant works are given below: das and chakraborty (2018) proposed lin-earsvm using tf-idf along with next word negation (nwn) for sentiment classification of three different datasets (movie review dataset, product review dataset, and sms spam dataset). they obtained accuracies of 89.91%, 88.86%, and 96.83% on imdb review datasets, amazon product review, and sms spam datasets respectively. to perform sa in code-mixed dravidian languages (tamil and malayalam) and english language, hegde and shashirekha (2022) used long short term memory (lstm) model trained with dynamic meta embedding (dme) and obtained f1-scores of 0.36, 0.74, and 0.37 for tamil, english, and malayalam languages respectively. balouchzahi and shashirekha (2020) to solve the arabic sentiment classification problem, @xcite used various hyperparameter tuning techniques on six ml algorithms (lr, svm, dt, ridge classifier, random forest (rf), and naive bayes (nb)) trained using tf-idf of word unigrams. all algorithms were experimented with and without the hyperparameter tuning process and among all the algorithms, they obtained highest accuracy for svm with an accuracy of 95.62% using bayesian optimization. @xcite proposed multitask learning for offensive language detection (old) and sa and experimented on the dataset kannada code-mixed dataset (kancmd) created by scraping the youtube comments. ml algorithms (dt, lr, svm, mnb, k-nearest neighbors (k-nn), and rf) are trained with tf-idf of n-grams in the range (1, 3) for both old and sa tasks. the rf model outperformed other models with macro f1-scores of 0.59 and 0.66 for sa and old respectively. poornima and priya (2020) proposed ml classifiers (mnb, svm, and lr) trained with word bigrams for sa of malayalam tweets and obtained an accuracy of 86.23% for lr classifier. in spite of several approaches explored for sa, the performance of many of the approaches are still low for low-resource languages, indicating the need to explore models for sa in to improve the performance. ## methodology the proposed methodology for sa in code-mixed tamil and tulu (dravidian languages) includes three major steps: pre-processing, feature extraction, and classifier construction. the framework of the proposed methodology is shown in figure 1 and the steps are briefly explained below: ## experiments and result the statistics of code-mixed tamil @xcite and tulu @xcite datasets shared by the organisers of the shared task in shown in the performance of the proposed model for development (dev) set and test sets for both the languages are shown in for code-mixed tamil and tulu datasets respectively. the comparison of the macro f1-scores of other participants' models with our proposed model for tamil and tulu datasets are shown in figure 2 and 3 respectively. the misclassified samples of the tulu test test along with actual and predicted labels and remarks, for linearsvc model are shown in ## conclusion in this paper, our team mucsd describes the three proposed models, namely: lr, linearsvc, and an ensemble (lr, dt, and svm) classifier with hard voting, trained with tf-idf of word unigrams, to perform sa of tulu and tamil code-mixed texts. these models are submitted to the "sentiment analysis in tamil and tulu" shared task at dravidian-langtech@ranlp 2023. among the proposed models, linearsvc achieved f1-scores of 0.189 and 0.508 and secured 8 th and 9 th rank for tamil and tulu texts respectively. different feature sets and different classifications will be investigated to improve the performance of the proposed models in the future.