VEL @ D ravidian L ang T ech: Sentiment Analysis of T amil and T ulu (2023)

Abstract: We participated in the Sentiment Analysis in Tamil and Tulu - DravidianLangTech 2023-RANLP 2023 task in the team name of VEL. This research focuses on addressing the challenge of detecting sentiment analysis in social media code-mixed comments written in Tamil and Tulu languages. Code-mixed text in social media often deviates from strict grammar rules and incorporates non-native scripts, making sentiment identification a complex task. To tackle this issue, we employ pre-processing techniques to remove unnecessary content and develop a model specifically designed for sentiment analysis detection. Additionally, we explore the effectiveness of traditional machine-learning models combined with feature extraction techniques. Our best model logistic regression configurations achieve impressive macro F1 scores of 0.43 on the Tamil test set and 0.51 on the Tulu test set, indicating promising results in accurately detecting instances of sentiment in code-mixed comments.

## introduction the constant increase in web and social media usage by the global audience has resulted in a substantial increase in the quantity of textual data expressing opinions @xcite . this publicly available textual data contains valuable insights that can be utilized in numerous disciplines, including marketing, finance, politics, and security @xcite . these insights provide an excellent opportunity for individuals and businesses to discover the opinions of a user base, allowing them to make informed decisions regarding the enhancement of their brands and services @xcite . however, as more opinionated data becomes available on the internet, it becomes exceedingly difficult to read and trace emotions. this issue can be addressed using techniques of sentiment analysis @xcite . sentiment analysis is regarded as the computational study of sentiments, opinions, emotions, and evaluations in order to comprehend better a person's reactions and attitudes towards multiple entities @xcite . due to its prospective applications, the computational treatment of opinion, sentiment, and subjectivity has recently garnered a great deal of attention. sentiment analysis identifies the perspective(s) underlying a text segment; an example application is classifying a movie review as "thumbs up" or "thumbs down." the situation is more complicated for low-resource languages like tamil and tulu due to the poor performance of multilingual or language-specific models and the lack of adequate benchmark datasets @xcite @xcite . tools and models for sentiment analysis have undergone substantial development over the course of several years, particularly for european languages. on the other hand, comparable tools for dravidian languages are difficult to come by @xcite . this is due to the fact that cuttingedge pre-processing tools for indian languages, such as pos taggers, shallow parsers, and other similar programs, are not easily accessible @xcite @xcite . it has been observed that people in bilingual and multilingual communities often combine a number of languages when they are speaking or writing informally @xcite . younger generations are increasingly using dravidian code-mixed languages in a variety of contexts, including but not limited to advertising, entertainment, and social media @xcite . examples of these languages include malayalam and tamil. roman script is the most prevalent form used to write the language (mandalam and sharma, 2021; @xcite . we participated in sentiment analysis shared tasks for tamil and tulu @xcite , which focused on detecting sentiment in social media comments. the task was organized by dra-vidianlangtech 2023. using the provided dataset, we developed three machine-learning models that utilized tf-idf feature extraction. among our models, the logistic regression model yielded the best results, achieving a macro f1 score of 0.43 for tamil and a macro f1 score of 0.51 for tulu. these scores indicate the effectiveness of our approach in accurately identifying instances of sentiment in social media comments for the tamil and tulu languages. our participation in this shared task has provided valuable insights into detecting and understanding discriminatory behavior on online platforms. ## related work initially, sentiment analysis relies heavily on artificial feature selection and a sentiment dictionary for classification @xcite . three levels of sentiment analysis were investigated: document, sentence, and aspect. to categorize the prevailing sentiment of the entire document and sentence, the tasks at the document and sentence levels are virtually identical. on the other hand, the objective at the facet level is to identify opinions expressed about entities and aspects of entities. earlier research proposed a number of sentiment analysis methods. there are three categories of these approaches: (i) lexicon-based, (ii) machine learning, and (iii) hybrid. the vast majority of research in this area concentrates on a lexicon-based approach that compiles sentiment words. all sentiment analysis tasks require machine learning approaches, the majority of which are either supervised or semi-supervised. a supervised approach is a common solution for sentimentrelated tasks. on the basis of labeled datasets, it is frequently employed to ascertain the polarity of sentiments. the advantage of this method is its capacity to adapt and build efficiently trained models for a specific domain. the primary disadvantage of this method, however, is its inefficiency when applied to diverse data across domains. this is due to domain dependence and data scarcity in specific domains. since 2011, deep learning has been utilized in the field of sentiment analysis research. ## task details sentiment analysis is the task of identifying subjective opinions or emotional responses about a given topic. it has been an active area of research in the past two decades in academia and industry. there is an increasing demand for sentiment detection on social media texts which are largely code-mixed for dravidian languages. code-mixing is a prevalent phenomenon in a multilingual community, and code-mixed texts are sometimes written in nonnative scripts. systems trained on monolingual data fail on code-mixed data due to the complexity of code-switching at different linguistic levels in the text. the shared task -a presents a new gold standard corpus for sentiment detection of codemixed text in dravidian languages (tamil-english, and tulu-english) 1 . the goal of this task is to identify the sentiment polarity of the code-mixed dataset of comments/posts in tamil-english and tulu-english collected from social media. the dataset description was tabulated in the ## methodology our methodology for the sentiment analysis shared task involved implementing three traditional machine-learning models and performing several data processing steps. we started by importing essential packages such as pandas, numpy, and nltk foot_1 with sklearn library foot_2 . these packages to begin, we read the train, development, and test files using pandas, which contained the text data used for training and evaluating the sentiment analysis models. we then proceeded to clean the files by removing unwanted links, '@' symbols, and other characters that could potentially impact the accuracy of our models. tokenization was performed using the nltk package, breaking down the text data into individual words or terms. lowercasing the contents of the files was another important step to ensure consistency, as it helped improve the models' ability to detect sentiment analysis accurately. once the data was preprocessed, we used the term frequency -inverse document frequency (tfidf) vectorizer method to transform the text data into a numeric representation that could be utilized by our machine-learning models. the models employed for training were logistic regression, multinomial naive bayes, and linear svc. we trained these models using the preprocessed train data. to evaluate their performance, we utilized the development data, analyzing metrics such as the confusion matrix foot_3 and classification report foot_4 . the macro f1 scores obtained from the development data allowed us to determine the best-performing model. in the final stage, we assessed the selected best model's performance using the test data. this step provided insights into the model's generalizability and effectiveness in handling new instances of sentiment analysis. our comprehensive methodology ensured a robust approach to sentiment analysis in social media comments, leveraging traditional machine-learning techniques and effective data processing steps. ## results and discussion in our evaluation of the machine learning models for sentiment analysis, we employed several performance metrics, including accuracy (acc), macro precision (m p), macro recall (m r), macro f1 (m f1), weighted precision (w p), weighted recall (w r), and weighted f1 (w f1) scores. our experimentation involved three different models: logistic regression (lr), multinomial naive bayes (mnb), and linear support vector classification (l-svc), all utilizing tf-idf feature extraction. for the tamil language, the macro f1 scores obtained for the models were as follows: logistic regression achieved a score of 0.43, multinomial naive bayes obtained 0.20, and linear support vector classification achieved 0.41. among these models, logistic regression performed particularly well in accurately identifying sentiment in tamil social media comments when using the tf-idf vectorizer. the overall classification report is tabulated in moving on to the tulu language, we obtained the following macro f1 scores: logistic regression achieved a score of 0.51, multinomial naive bayes obtained 0.25, and linear support vector classification achieved 0.49. once again, logistic regression demonstrated superior performance in accurately detecting sentiment in tulu social media comments when utilizing tf-idf feature extraction. the overall classification report is tabulated in these results indicate that the logistic regression model outperformed the other models across both tamil and tulu languages in terms of macro f1 scores. the tf-idf vectorizer played a crucial role in capturing important features and enabling accurate sentiment analysis. overall, our experiments show the effectiveness of logistic regression coupled with tf-idf feature extraction for sentiment analysis in social media comments for both tamil and tulu languages. these findings highlight the potential of machine learning techniques in identifying sentiment and understanding discriminatory ## conclusion this study focused on the detection of sentiment analysis in the tamil and tulu languages using machine learning models. by employing traditional models with tf-idf feature extraction, we achieved impressive results, with a macro f1 score of 0.43 on the tamil test set and 0.51 on the tulu test set. these findings underscore the effectiveness of our machine learning approach in accurately classifying sentiment analysis instances. we conclude that the tf-idf with logistic regression (lr) model is a suitable choice for this task, offering potential applications in addressing social issues and promoting inclusivity in online spaces. future research can further improve the model's performance through fine-tuning techniques and larger datasets. this study contributes to advancing sentiment analysis and highlights the importance of accurate sentiment classification in understanding and addressing social dynamics in multilingual online platforms.