Poorvi@ D ravidian L ang T ech: Sentiment Analysis on Code-Mixed T ulu and T amil Corpus (2023)

Abstract: Sentiment analysis in code-mixed languages poses significant challenges, particularly for highly under-resourced languages such as Tulu and Tamil. Existing corpora, primarily sourced from YouTube comments, suffer from class imbalance across sentiment categories. Moreover, the limited number of samples in these corpus hampers effective sentiment classification. This study introduces a new corpus tailored for sentiment analysis in Tulu code-mixed texts. The research applies standard pre-processing techniques to ensure data quality and consistency and handle class imbalance. Subsequently, multiple classifiers are employed to analyze the sentiment of the code-mixed texts, yielding promising results. By leveraging the new corpus, the study contributes to advancing sentiment analysis techniques in under-resourced code-mixed languages. This work serves as a stepping stone towards better understanding and addressing the challenges posed by sentiment analysis in highly under-resourced languages.

## introduction online social media material is expanding at an exponential rate. social media platforms allow users to freely express themselves in their native languages thanks to their multilingual user interface. as a result, a linguistic phenomenon known as code-mixing in social media data has become prevalent, attracting the interest of academics in disciplines like sociolinguistics and natural language processing (nlp). the informality of code-mixed text, however, presents a number of difficulties, including those with data extraction and summarization. sentiment analysis has been a significant research field in the field of code-mixed data analysis in recent years @xcite @xcite . india has the greatest population of speakers of english as a second language thanks to its rich linguistic past and close relationship with english. native indian language speakers don't use unicode while exchanging information on social media platforms. they employ code-mixing to combine latin script with english words or phrases in their original language to communicate themselves @xcite . additionally, there has been work done on sentiment analysis on youtube comments @xcite . it is difficult to process these natural languages for diverse language-processing tasks @xcite . compared to other languages, the regional languages of india are thought to have few resources @xcite . this paper introduces a novel methodology for sentiment analysis on code-mixed tulu and tamil corpora, considering the challenges associated with class imbalance. an additional corpus is curated by scraping youtube comments on tulu videos, enriching the code-mixed tulu corpus and providing more comprehensive resources for analysis. this paper proposes a new stopwords list, tailored for both english and tulu languages, and utilizes a synonyms list to address inconsistent spelling variations in the corpus. these contributions advance the field of sentiment analysis on code-mixed languages, offering insights and guidance for effective analysis and improving the overall performance of sentiment classification models. ## related work there is a lot of research being done right now on code mixing in natural language processing (nlp) jobs. in their thorough investigation of the difficulties code-mixed nlp faces in a multilingual society, srivastava et al. @xcite shed light on the current state of this field's nlp research. the unique unified approach put out by choudhary et al. @xcite aims to overcome the drawbacks associated with using codemixed text in nlp. their method includes a preprocessing step that groups distinct word variations based on an empirical similarity measure, making analysis and processing more efficient. using datasets created expressly to show codemixing between bengali, english, and hindi, barman et al. @xcite report an ongoing research project on automatic language recognition on social media platforms. according to their first results, a dictionary-based strategy outperforms supervised classification and sequence labelling techniques in solving this issue. by creating a new code-switched dataset for hindi-english language pairings and carrying out a comparative evaluation of conventional machine learning models for word-level language recognition, mave et al. @xcite ) make a contribution to the area. their research offers insightful information about the performance of these models in contexts with code-mixed linguistics. the following five language pairs underwent neural machine translation by vyawahere et al @xcite : kannada to tamil, kannada to telugu, kannada to malayalam, kannada to sanskrit, and kannada to tulu. the datasets for each of the five language pairings were used to train a variety of translation models, including seq2seq models like lstm, bidirectional lstm, conv2seq, and state-of-the-art transformers from scratch. although much research has been done on sa in the english language, data on the web also offers information in various other languages that should be examined. the goal of shah et al. @xcite is to analyse, assess, and debate the methodologies, algorithms, and difficulties encountered by the researchers when conducting the sa on indigenous languages. in today's age, twitter contains a vast array of emotions and viewpoints. it offers a significant volume of sentiment-related information, but extracting data from twitter necessitates appropriate techniques. the study conducted by rakshitha et labels train set test set positive 3487 344 negative 736 60 mixed feelings 1094 107 neutral 1921 197 ## corpus details in the case of the code-mixed tulu corpus, a combination of the existing corpus @xcite and the newly created dataset proposed in this paper was utilized. for the code-mixed tamil corpus, the data was obtained from the established dataset (chakravarthi et al., 2020). ## data pre-processing the corpus underwent a series of pre-processing steps prior to the application of models. initially, emojis were replaced with their corresponding names. following this, a sequence of transformations was performed, which involved the removal of html tags, urls, punctuation marks, special characters, numbers, and excessive whitespace. additionally, the text was converted to lowercase to ensure uniformity. stop words, however, were not removed from the corpus, as their exclusion resulted in a decrease in performance. to establish consistent spelling, a predefined list of synonyms that i created was utilized to replace words with their appropriate alternatives. moreover, a label encoder was applied to the annotation labels, facilitating ease of use for the models during training @xcite . the tf-idf vectorizer was employed with specific parameters, including a maximum of 5000 features and an n-gram range of (1, 2) @xcite . in this context, the use of inverse document frequency (idf) was disabled to optimize vectorisation. to expand the corpus, the textattack easy-dataaugmenter technique was utilized, resulting in a quadrupling of the corpus size. this augmentation process helped to introduce additional variations in the data, thereby enhancing the overall model performance @xcite . considering the imbalanced distribution of positive comments within the corpus, the smote (synthetic minority over-sampling technique) algorithm was employed. this technique ensured an equal representation of samples from each class, ultimately improving the performance of the models. class imbalance is a common challenge in many machine learning applications, particularly in sentiment analysis tasks. one effective approach to address this issue is synthetic minority oversampling technique (smote). smote is a data augmentation technique specifically designed to tackle class imbalance by generating synthetic samples for the minority class @xcite . the method works by identifying minority class instances and creating synthetic examples along the line segments connecting them. this process increases the diversity of the minority class and helps to balance the class distribution in the corpus. by introducing synthetic samples, smote not only mitigates the impact of class imbalance but also improves the overall performance of classification models. it allows the classifier to learn from a more balanced representation of the data, leading to enhanced predictive capabilities. moreover, smote is widely applicable across various machine learning algorithms and has proven to be particularly effective in sentiment analysis tasks, where imbalanced sentiment classes are often encountered. these pre-processing and data augmentation techniques collectively contributed to the refine-ment and enrichment of the corpus, thereby facilitating more accurate sentiment analysis results. in the study, a range of models @xcite were employed to analyze the corpus, and their parameters were fine-tuned using 5-fold crossvalidation. the application of cross-validation ensures robust evaluation and provides insight into the models' performance. the chosen parameters aim to optimize the models' predictive capabilities while maintaining a balance between bias and variance. the details of the models considered in the analysis are given below. ## experiments and results a series of experiments were conducted to identify the optimal configuration for sentiment analysis on the code-mixed corpus. the evaluation of the classification system's performance was based on the weighted averaged f-score, which provides a comprehensive measure across all classes. to ensure reliable results, 5-fold cross-validation was employed to determine the best parameters for the models. the corpus used for analysis exhibited an imbalance among the classes, necessitating the implementation of the synthetic minority over-sampling technique (smote). this technique effectively addressed the class imbalance issue and led to significant improvements in the performance of most models. moreover, data augmentation techniques were employed using the textattack library. this approach further enhanced the corpus by generating additional samples, contributing to the overall performance improvement of the models. after thorough experimentation and analysis, the stacking classifier, specifically the combination knearest neighbours, random forest, and multinomial naive bayes with logistic regression as the meta-classifier, emerged as the best model for sentiment analysis on the code-mixed tulu language. in contrast, logistic regression alone demonstrated superior performance for sentiment analysis on the code-mixed tamil language. these findings highlight the effectiveness of the proposed models and the significance of addressing class imbalance and utilizing data augmentation techniques in code-mixed sentiment analysis tasks. ## conclusion in conclusion, this paper presents a methodology for sentiment analysis on a code-mixed corpus consisting of tulu and tamil languages extracted from youtube comments. the unique characteristics of code-mixed data, such as inconsistent spelling and the absence of stemming and lemmatisation libraries, pose challenges for traditional classifiers. this study looks at various classifiers and their performance of the code-mixed corpora. however, despite achieving notable performance, there remains ample room for further improvement in prediction accuracy. this study highlights the potential for future research endeavours to enhance sentiment analysis techniques specifically tailored for codemixed languages.