KEC _ AI _ NLP @ D ravidian L ang T ech: Sentiment Analysis in Code Mixture Language (2023)

Abstract: Sentiment Analysis is a process that involves analyzing digital text to determine the emo- tional tone, such as positive, negative, neu- tral, or unknown. Sentiment Analysis of code- mixed languages presents challenges in natural language processing due to the complexity of code-mixed data, which combines vocabulary and grammar from multiple languages and cre- ates unique structures. The scarcity of anno- tated data and the unstructured nature of code- mixed data are major challenges. To address these challenges, we explored various tech- niques, including Machine Learning models such as Decision Trees, Random Forests, Lo- gistic Regression, and Gaussian Na ̈ıve Bayes, Deep Learning model, such as Long Short- Term Memory (LSTM), and Transfer Learning model like BERT, were also utilized. In this work, we obtained the dataset from the Dravid- ianLangTech shared task by participating in a competition and accessing train, development and test data for Tamil Language. The results demonstrated promising performance in senti- ment analysis of code-mixed text. Among all the models, deep learning model LSTM pro- vides best accuracy of 0.61 for Tamil language.

## introduction sentiment analysis, also referred to as opinion mining, is a computational process that utilizes natural language processing (nlp), text analysis, and computational linguistics to uncover the emotional sentiment expressed in each text. it aims to categorize and determine opinions regarding a product, service, or idea, by extracting polarity (positivity or negativity), subject matter, and the opinion holder within the text. sentiment analysis can be performed on various levels, including full documents, paragraphs, sentences, or even smaller units. it finds applications in diverse domains such as product reviews, social media analysis, and market research. different types of sentiment analysis exist, such as aspect-based sentiment analysis, grading sentiment analysis, multilingual sentiment analysis, and emotion detection. approaches for sentiment analysis include knowledge-based techniques, statistical methods, and machine learning algorithms. social media platforms serve as significant sources of data for sentiment analysis, as they generate vast and interconnected information in the form of user-generated content. this study presents recurrent convolutional neural networks and machine learning techniques for text classification tasks, combining the strengths of both recurrent and convolutional neural networks to capture long-term dependencies. ## methodology machine learning models have revolutionized the field of artificial intelligence by enabling computers to learn and make predictions or decisions without being explicitly programmed. in this introduction, we will explore several popular machine-learning models and their applications in sentiment analysis. the proposed system workflow is presented in figure 1 . decision trees are a powerful supervised learning technique that can be used for both classification and regression tasks. they create a tree-like model where each internal node represents a feature, each branch represents a decision rule, and each leaf node represents an outcome or prediction. decision trees are intuitive, easy to interpret, and can handle both numerical and categorical data. they are often used in sentiment analysis to classify text into positive, negative, or neutral sentiments. random forests are an ensemble learning method that combines multiple decision trees to make more accurate predictions. it works by constructing a multitude of decision trees, each trained on a different subset of the training data and using random feature subsets. the final prediction is made by aggregating the predictions of individual trees. random forests are known for their robustness, ability to handle high-dimensional data, and resistance to overfitting. they are commonly used in sentiment analysis to improve classification accuracy. logistic regression is a popular classification algorithm used to predict the probability of a binary outcome based on input variables. despite its name, logistic regression is a linear model, but it applies a logistic function to the linear output to transform it into a probability. it is widely used in sentiment analysis to determine the likelihood of a given text belonging to a specific sentiment class. logistic regression is computationally efficient and can provide interpretable results. gaussian naïve bayes is a probabilistic classifier based on baye's theorem and assumes independence between features. gaussian naïve bayes specifically assumes that the features follow a gaussian distribution. it is a simple yet effective algorithm that works well with high-dimensional data and requires a small amount of training data. gaussian naïve bayes is often used in sentiment analysis to classify text based on the likelihood of belonging to a particular sentiment class. performing sentiment analysis using machine learning models are depicted in figure 2 . deep learning models, such as lstm, are a type of artificial neural network [7-11] with multiple layers that can learn complex patterns and dependencies in data. lstm networks are specifically designed to capture long-term dependencies by using memory cells and gates that regulate the flow of information. lstms have been highly successful in natural language processing tasks, including sentiment analysis. they can learn contextual information, understand sequential patterns, and effectively model text sentiment over longer sequences. performing sentiment analysis by lstm is represented in figure 3 . transfer learning -bert (bidirectional encoder representations from transformers) is a state-of-the-art deep learning model for natural language processing tasks. it is based on the transformer architecture and is pre-trained on a large corpus of text data. bert has achieved remarkable results in various nlp tasks, including sentiment analysis, by leveraging its ability to understand contextualized word representations. transfer learning with bert involves fine-tuning the pre-trained model on a specific sentiment analysis task, which can significantly improve performance even with limited training data. sentiment analysis by bert is presented in figure 4 . ## performance evaluation whether utilising machine learning or deep learning techniques, performance evaluation for sentiment analysis models often incorporates multiple indicators to gauge the model's efficacy. following are a few typical evaluation measures for sentiment analysis: accuracy: the ratio of examples that figure 4: sentiment analysis by bert ## conclusion in this research work, sentiment analysis is performed using machine learning, deep learning and transfer learning techniques, specifically gaussian naive bayes, decision trees,random forests,bert, and lstm. for the machine learning approach, we employed logistic regression, decision tree, and random forest algorithms, while for the deep learning approach, we utilized lstm models and for transfer learning approach, we utilized bert. after evaluating the models, we obtained an accuracy of 42 percent for the decision tree and 61 percent for the lstm. in summary, the lstm model outperformed the decision tree algorithm's precision, achieving a higher accuracy of 61 percent for sentiment analysis. however, the performance can be further improved by fine-tuning hyperparameters, exploring different architectures, or incorporating ensemble methods.