MUCS @ D ravidian L ang T ech2023: Sentiment Analysis in Code-mixed T amil and T ulu Texts using fast T ext (2023)

Abstract: Sentiment Analysis (SA) is a field of computational study that focuses on analyzing and understanding people’s opinions, attitudes, and emotions towards an entity. An entity could be an individual, an event, a topic, a product etc., which is most likely to be covered by reviews and such reviews can be found in abundance on social media platforms. The increase in the number of social media users and the growing amount of user-generated code-mixed content such as reviews, comments, posts etc., on social media have resulted in a rising demand for efficient tools capable of effectively analyzing such content to detect the sentiments. However, SA of social media text is challenging due to the complex nature of the code-mixed text. To tackle this issue, in this paper, we team MUCS, describe learning models submitted to “Sentiment Analysis in Tamil and Tulu” -DravidianLangTech@Recent Advances In Natural Language Processing (RANLP) 2023. Using fastText embeddings to train the Machine Learning (ML) models to perform SA in code-mixed Tamil and Tulu texts, the proposed methodology exhibited F1 scores of 0.14 and 0.204 securing 13th and 15th rank for Tamil and Tulu texts respectively.

## introduction in this digital era, social media platforms have become an integral part of the life of many people, especially the younger generation and have impacted people's perception of networking and socialising to a greater extent @xcite @xcite . this concept has influenced people to communicate efficiently and quickly using various social media platforms and has resulted in the increase in large amount of user-generated text data in the form of posts, comments, opinions, emotions, attitudes and re-views, making them a best source for user sentiments @xcite . identifying the sentiments of these text as positive, negative, neutral, etc., is the objective of sa as it is useful for various applications @xcite @xcite . for example, sa can be used to determine which videos are liked by people on youtube, based on the words/phrases in the comments for the video. sa can also help to determine whether a user is happy, sad, or angry, with the video. as there is no barrier of language and content on social media, users feel convenient to post comments very informally by mixing words and sentences of more than one language (usually with one language being english) in more than one script, usually the native script and roman script. further, due to the limitations of keyboard/keypad in computers/smart phones, users find it easy to key in the posts/comments in roman script @xcite @xcite . this phenomena of mixing the linguistic units of more than language in one utterance or text is called as code-mixing and it has almost become the official language of social media due to the increased number of users using this language @xcite . analyzing the user sentiments in code-mixed language is challenging due to inadequate resources and tools to address the text in code-mixed language. the complexity of the task increases if the code-mixed text is in low-resource languages such as kannada, tulu, tamil, malayalam, etc. as the code-mixed language is free from the grammar of any languages, users create words/sentences according to their whims and fancies which makes it interesting and challenging to analyze such texts. to address the challenges of processing codemixed dravidian languages for sa, in this paper, we -team mucs, describe the models submitted to "sentiment analysis in tamil and tulu" shared task at dravidianlangtech@ranlp 2023 1 @xcite . the shared task consists of a message-level polarity classification task for sa in code-mixed tamil @xcite and tulu @xcite . given a youtube comment in tamil/tulu, the aim of the shared task is to develop models to classify the given comment into positive, negative, neutral, or mixed emotions. few tulu and tamil comments in romanized script and their english translations are shown in a the rest of the paper is organized as follows: section 2 contains related work and section 3 describes the methodology. section 4 describes the experiments and results followed by conclusion and future work in section 5. ## related work many of the techniques explored by researchers for sa focus on high-resource languages like english, spanish etc. off late, sa is also being explored in code-mixed low-resource languages. description of some of the relevant sa works in code-mixed low-resource languages are given below: cosad -a code-mixed sa model for dravidian languages proposed by @xcite makes use of char n-grams, char sequences, and syllables, to train an ensemble (linear support vector machine (lsvm), logistic regression (lr) and multi-layer perceptron (mlp) classifers) model with majority voting to identify sentiments in codemixed kannada, malayalam, and tamil languages. their models obtained average weighted f1-scores of 0.628, 0.726,and 0.619 for code-mixed kannada, malayalam, and tamil languages respectively. ensemble of random forest (rf), multi-layer perceptron (mlp) and gradient boosting is proposed by @xcite to identify hate speech and offensive content in monolingual english, hindi, and marathi languages and code-mixed english-hindi language pairs. these ensemble models trained using a combination of the term frequency -inverse document frequency (tf-idf) of word uni-grams, character n-grams in the range (2, 3), pre-trained word embeddings (word2vec), hashtag embeddings (hastagvec), and emo2vec embeddings, obtained f1 scores 0.8251, 0.6323, 0.7830 and 0.6721 for english, hindi, marati and codemixed english-hindi language pair respectively. sa of youtube comments in code-mixed tamil, malayalam and kannada language explored by babu and eswari (2021) using paraphrase crosslingual language model-robustly optimized bidirectional encoder representations from transformers (xlm-roberta) trained with hyperparameters (@xmath0, learning @xmath1, @xmath2, and @xmath3.5.) obtained f1-scores of 0.71, 0.75 and 0.62 on tamil, malayalam and kannada languages respectively. chanda and pal (2020) experimented feature extraction using bidirectional encoder representations from transformers (bert), distilbert and fasttext to train the lr classifiers to perform sa in code-mixed tamil and malayalam languages. among these models, lr model trained with fasttext embeddings outperformed other models with f1 scores of 0.58 and 0.63 for code-mixed tamil and malayalam languages respectively. xlm-roberta fine-tuned on code-mixed malayalam and tamil texts by bai et al. (2021), to automatically detect sentiments, achieved f1 scores of 0.804 and 0.676 for malayalam and tamil languages respectively. convolutional neural networks and bi-directional long short-term memory (cnn+bi-lstm) model trained with fasttext and glove pre-trained models by mengistie and kumar (2021) for sa of covid-19 public reviews achieved 99.33 and 97.55 accuracy. zhu and dong (2020) proposed sa of dravidian codemixed text using multilingual bidirectional encoder representations from transformer (mbert) model and used self-attention to assign a weight to the output of the bilstm. their models achieved f1-scores of 0.73 and 0.64 for malayalam and tamil languages respectively. from the literature review, it is evident that even though there are many models for sa of codemixed low-resource languages, very few works have been reported for sa of code-mixed tamil language and no work has been reported for the sa of code-mixed tulu language. hence, there is lot of scope to develop sa models for code-mixed low-resource tulu and tamil languages. ## methodology the proposed methodology for sa in code-mixed tulu and tamil includes: text augmentation, preprocessing, feature extraction, and model construction. the framework of the proposed methodology is shown in figure 1 and the steps are explained below: ## experiments and results several experiments were conducted by combining various features and classifiers. the combination of features and classifiers which gave good performance on the development sets are used to train the proposed models. the proposed models are evaluated on the test set and the predictions are assessed by the organizers based on macro f1-score for the final evaluation and ranking. the performance of the proposed models for both tamil and tulu datasets are shown in the results illustrate that, ensemble model exhibited better performance over the other model with macro f1 score of 0.13 for tamil text. even though text augmentation is used to increase the samples of the minority classes (3 classes in tamil and 2 classes in tulu) to some extent, the datasets still remains imbalanced. tamil dataset has a very large difference between the number of samples in 'positive' class and other classes where as the difference between the number of samples in 'positive' class and other classes in tulu dataset is comparatively less. this clearly indicates the effect of data imbalance on the performance of the classifiers. ## conclusion this paper describes the models submitted to "sentiment analysis in tamil and tulu" -dravidian-langtech@ranlp 2023 shared task. the proposed methodology consists of balancing the imbalance data using text augmentation, using fasttext embeddings and tf-idf as features to train svm and ensemble model (lr, bnb and svc models) with hard voting to perform sa. the proposed models exhibited f1 scores of 0.14 and 0.20 securing 13 th and 15 th rank for tamil and tulu datasets respectively.