Sentiment Analysis in Code-Mixed T elugu- E nglish Text with Unsupervised Data Normalization (2021)

Abstract: In a multilingual society, people communicate in more than one language, leading to Code-Mixed data. Sentimental analysis on Code-Mixed Telugu-English Text (CMTET) poses unique challenges. The unstructured nature of the Code-Mixed Data is due to the informal language, informal transliterations, and spelling errors. In this paper, we introduce an annotated dataset for Sentiment Analysis in CMTET. Also, we report an accuracy of 80.22% on this dataset using novel unsupervised data normalization with a Multilayer Perceptron (MLP) model. This proposed data normalization technique can be extended to any NLP task involving CMTET. Further, we report an increase of 2.53% accuracy due to this data normalization approach in our best model.

## introduction in recent times, huge volumes of data are being generated on social media and online blogging. the usage of multiple languages in day-to-day conversations and the minimal linguistic restrictions on the online content lead to the generation of code-mixed data. code-mixing or code-switching is the usage of two or more languages in a single sentence or a conversation. the generated code-mixed data can be used to extract potential knowledge like emotion, news or information. sentimental analysis on code-mix social media data helps us to understand the underlying sentiment of the phrase or sentence, which can have many practical use cases in the real world. for example, it can be used to understand the sentiment of restaurant or movie reviews. code-mixed data can be easily extracted from various online platforms like social media and blogs using apis and various web-scraping tools. in spite of having huge amounts of data, performing sentiment analysis on it is still challenging because of its unstructured and noisy nature @xcite , @xcite , @xcite ). hence, this requires robust preprocessing techniques to normalize this unstructured data. in this paper, we have performed sentimental analysis on code-mixed telugu-english text (cmtet) with an unsupervised data normalization. we analyse and classify each sentence into three sentiments, positive, negative and neutral. to the best of our knowledge this is the first work to propose a framework to perform sentiment analysis on cmtet. the reason for choosing english as our secondary language for the code-mixed data is because we observed that most of the telugu data available in social media is very often code-mixed with english. the rest of the paper is organized as follows. in section 2, we discuss the existing sentiment analysis approaches in other code-mixed languages. section 3 introduces the dataset and describes the methodology involved in preprocessing and annotating this dataset. in section 4, we discuss in detail various challenges faced in performing sentiment analysis in cmtet. in section 5, we discuss about the proposed data normalization technique for cmtet. in section 6, we explain the process of feature extraction and sentiment classification. in section 7, we show the results of the proposed methods and make a comparative study on the accuracy of all those models. in section 8, we discuss the problems faced by the models in predicting the sentiment. section 9 concludes the paper with the summary and scope for future work. ## related work for the task of sentiment analysis on code-mixed hindi-english text, @xcite used a lexicon-based approach on the fire 2013 @xcite and fire 2014 datasets. the dataset consisted of the user comments from public facebook pages of two of the most popular celebrities. @xcite introduced a new dataset and used subword-lstm to address the noisy nature of the text and reported an improvement of 18% on the baseline. choudhary et al. (2018) proposed clustering of code-mixed word variations with skip-gram vectors based on similarity. they also used contrastive learning and projected the code-mixed sentences into a common sentiment space using shared parameters. sukhpreet kaur (2021) used attention based models with word-level, sub-word level and character-level representations of the sentences, and reported that bi-lstm performed the best. chakravarthi et al. (2020) had presented an annotated code-mixed corpus in malayalam-english language of youtube comments for the task of sentiment analysis. @xcite performed sentiment analysis in code-mixed malayalam-english and tamil-english text using awd-lstm @xcite and reported a weighted f1-score of 0.6 on both the datasets. @xcite proposed an ensemble model to perform sentiment analysis in hindi -english and bengali -hindi -english corpus. the ensemble model is built on linear svm, logistic regression and random forest based on a voting classifier. @xcite created a persian-english code mixed data corpus from tweets. they also proposed a bi-lstm based ensemble model which uses bert embeddings and translation models to learn sentiment of tweets. yadav and chakraborty (2020) proposed a zero-shot approach to solve sentiment analysis task on code-mixed spanish-english text. they used multilingual and crosslingual embeddings to transfer knowledge from monolongual text to code-mixed text. they reported an increase of 3% in accuracy over the previous state-of-the-art. in the task of sentiment analysis on social media text, there is a lot of research on using different models to address the problem of its noisy nature. our work differs in this context, where, we propose an unsupervised approach for normalizing cmtet. and to the best of our knowledge, this is the first work of sentiment analysis in cmtet. ## dataset in this paper, we introduce a new dataset for sentiment analysis in cmtet. the methodology consists of three main phases: data collection, data cleaning, and data annotation. the below sections describe each phase in detail. ## challenges in cmtet the cmtet poses new challenges because of its unstructured nature due to the following phenomenons. ## data normalization in this section, we propose an unsupervised data normalization technique for normalizing cmtet. the architecture for data normalization can be found in fig 2 . as a first step, we performed elongation normalization and then used the language tags in the dataset to normalize telugu words and english words separately. we then spell-checked english words using a defined similarity metric with an english dictionary. for telugu words, we performed a two-stage normalization to normalize transliteration and spelling errors. the below sections explain this architecture in detail. ## method in this section, we focus on explaining our sentiment analysis pipeline. fig. 3 shows our end to end approach wherein we take raw data i.e. a sentence and perform data normalization (transliteration and spelling normalization) as explained in section 5. once we have this normalized data we perform feature extraction us-ing n-grams and term-frequency and inverse-document frequency (tf-idf) @xcite . these features are then passed to our sentiment classification models which outputs one of the labels namely positive, negative and neutral. we experimented on logistic regression (lr), naive bayes (nb), support vector machine (svm), random forest (rf) and multi layer perceptron (mlp) classification models to classify the final sentiment. ## experiments and results the motivation of the current work is achieve better sentiment analysis in cmtet using machine learning models. majority of the existing approaches majorly focus on single language sentiment analysis (zulkifli and lee, ## error analysis and observations the task of sentiment analysis on social media text is difficult in itself as it often contain sarcastic text. the other major challenge is the use of emoticons and emojis in social media text which often drive the sentiment of the sentence. ## conclusion and future work in this paper, we published a huge annotated dataset for sentiment analysis in cmtet to encourage further research. also, we presented a pipeline for this task with a novel data normalization technique. for each model, we have shown quantitatively that the proposed data normalization improves the overall performance across various metrics (precision, recall and f1-score) (refer although the current work addresses some of the most crucial challenges in sentiment analysis on cmtet it can further be extended to other languages. the proposed data normalization technique can also be leveraged in various other nlp tasks. we can further make the bench-marking more extensive by including other deep learning models like lstm, rnns and cnns.