MUCS @ D ravidian L ang T ech-2024: A Grid Search Approach to Explore Sentiment Analysis in Code-mixed T amil and T ulu (2024)

Abstract: Sentiment Analysis (SA) is a field of computational study that analyzes and understands people’s opinions, attitudes, and emotions toward any entity. A review of an entity can be written about an individual, an event, a topic, a product, etc., and such reviews are abundant on social media platforms. The increasing number of social media users and the growing amount of user-generated code-mixed content such as reviews, comments, posts etc., on social media have resulted in a rising demand for efficient tools capable of effectively analyzing such content to detect the sentiments. In spite of this, SA of social media text is challenging because the code-mixed text is complex. To address SA in code-mixed Tamil and Tulu text, this paper describes the Machine Learning (ML) models submitted by our team - MUCS to “Sentiment Analysis in Tamil and Tulu - Dravidian- LangTech” - a shared task organized at European Chapter of the Association for Computational Linguistics (EACL) 2024. Linear Support Vector classifier (LinearSVC) and ensemble of 5 ML classifiers (k Nearest Neighbour (kNN), Stochastic Gradient Descent (SGD), Logistic Regression (LR), LinearSVC, and Random Forest Classifier (RFC)) with hard voting trained using concatenated features obtained from word and character n-ngrams vectoized from Term Frequency-Inverse Document Frequency (TF-IDF) vectorizer and CountVectorizer. Further, Gridsearch algorithm is employed to obtain optimal hyperparameter values.The proposed ensemble model obtained macro F1 scores of 0.260 and 0.550 for Tamil and Tulu languages respectively.

## introduction sa is the process of examining opinions, emotions, and reviews to recognise the sentiments expressed by the users regarding a topic, movie, song, product, etc., available on online platforms @xcite . this user-generated content is used by businesses and individuals to gain knowledge and make well-informed decisions regarding their content @xcite . the user sentiments are usually available in codemixed language where words and/or sub-words belong to more than one language. processing the code-mixed user-generated content to develop sa models poses a significant challenge @xcite . this is especially notable when addressing sa in low-resource languages such as tulu, tamil, malayalam, and telugu @xcite . to address the challenges of detecting sa in usergenerated code-mixed low-resource languages, in this paper, we -team mucs, describe ml models submitted to the shared task "sentiment analysis in tamil and tulu -dravidianlangtech@eacl-2024" (s. @xcite . this shared task is modeled as a multi-class text classification problem with two distinct models: i) linearsvc and ii) ensemble of ml classifiers (knn, sgd, lr, lin-earsvc, and rfc) with hard voting, trained individually with the features obtained by the concatenation of tfidfvectorizer foot_0 and countvectorizer foot_1 of word and character n-grams, for sa in code-mixed tamil and tulu texts. in addition, the gridsearch method is used to find the ideal values for the hyperparameters of these classifiers. the rest of the paper is organized as follows: while section 2 describes the related works of sa, section 3 focuses on the description of the models submitted to the shared task followed by the experiments and results in section 4. the conclusion and future works are included in section 5. ## related work several ml models are experimented with various features for sa of user-generated content in code-mixed low-resource languages @xcite . some of the relevant works are outlined the related work reveals that the performances of sa models for code-mixed low-resource languages are still low, indicating the scope for developing models to improve the performance further. ## methodology the proposed methodology for sa in code-mixed tamil and tulu texts include: pre-processing, feature extraction (fe), and classifier construction. the framework of the proposed methodology is ## conclusion and future work this paper describes the models submitted by our team mucs to "sentiment analysis in tamil and tulu -dravidianlangtech@eacl-2024" shared task. the proposed methodology consists of using linearsvc and ensemble of ml classifiers with hard voting, trained individually with the features obtained by the concatenation of tfidfvectorizer and countvectorizer of word and character n-grams. further, in order to get the optimal hyperparameter values for these classifiers, the gridsearch method is used during training. the proposed ensemble models exhibited macro f1 scores of 0.260 and 0.550 securing 1 st and 2 nd ranks in the shared task for tamil and tulu languages respectively. suitable oversampling or text augmentation techniques will be explored further to improve the performance of the proposed models.