CUETS entiment S illies@ D ravidian L ang T ech- EACL 2024: Transformer-based Approach for Sentiment Analysis in T amil and T ulu Code-Mixed Texts (2024)

Abstract: Sentiment analysis (SA) on social media reviews has become a challenging research agenda in recent years due to the exponential growth of textual content. Although several effective solutions are available for SA in high-resourced languages, it is considered a critical problem for low-resourced languages. This work introduces an automatic system for analyzing sentiment in Tamil and Tulu code-mixed languages. Several ML (DT, RF, MNB), DL (CNN, BiLSTM, CNN+BiLSTM), and transformer-based models (Indic-BERT, XLM-RoBERTa, m-BERT) are investigated for SA tasks using Tamil and Tulu code-mixed textual data. Experimental outcomes reveal that the transformer-based models XLM-R and m-BERT surpassed others in performance for Tamil and Tulu, respectively. The proposed XLM-R and m-BERT models attained macro F1-scores of 0.258 (Tamil) and 0.468 (Tulu) on test datasets, securing the 2^{nd} and 5^{th} positions, respectively, in the shared task.

## introduction social media has changed how people network and socialize, especially the younger generation, and multilingual user interfaces allow people to express their emotions in their native languages @xcite @xcite . sentiment analysis (sa) may help firms assess their brand's image and sentiment and make informed customer relationship management and marketing choices. it analyzes social media postings to detect user attitudes @xcite . code-mixed texts greatly concern sentiment analysis. many multilingual societies use code-mixed texts, combining words, morphemes, and phrases from two or more languages @xcite . this behavior is problematic for sa systems, mainly when they utilize non-native scripts like roman letters to represent languages written in other scripts @xcite . coded language texts need specialized sentiment analysis due to language mixing and context-dependent emotions. scholars are improving security awareness methods to govern virtual communication's growth @xcite . the goal is to create a system that can classify code-mixed sentiment polarity in tamil-english and tulu-english code-mixed texts into four predetermined categories: positive, negative, mixed feeling, and neutral/unknown state. the main contributions of this study are: researchers studying several sa techniques tend to focus on high-resource languages such as english and spanish. however, sa is also being studied in code-mixed, low-resource languages. shetty (2023) trained various ml models for sa of tamil and tulu code-mixed texts. the proposed method yielded f1 scores of 0.14 and 0.204 in tamil and tulu, respectively. to detect abusive comments in code-mixed tamil text, bharathi and varsha (2022) employed bert, m-bert, and xlnet models. they obtained a weighted f1 score of 0.96 for tamil-english code-mixed text and a weighted f1 score of 0.59 for tamil text. babu and eswari (2021) improved sentiment analysis using paraphrase xlm-r on dravidian code-mixed youtube comments. they trained the model using tamil, malayalam, and kannada code-mixed language datasets and achieved f1 scores of 71.1, 75.3, and 62.5, respectively. chakravarthi et al. (2020a) created a gold standard tamil-english code-switched, sentiment-annotated corpus containing 15,744 comment posts from youtube. an m-bert-based model utilized by zhu and dong (2020) for sa where self-attention was employed to assign a weight to the output of the bil-stm. the proposed model achieved weighted average f1 scores of 0.73 and 0.64 in malayalam and tamil, respectively. rakshitha et al. (2021) proposed a model that used twitter apis to collect consumer reviews. textblob rated these reviews and classified them as favorable, negative, or neutral using a text classification algorithm. ehsan et al. (2023) developed bilstm network-based models for sentiment analysis of code-mixed tamil and tulu. elmo embedding was trained on larger unannotated code-mixed text corpora. the proposed model achieved macro f1-scores of 0.2877 and 0.5133 on tamil and tulu code-mixed datasets, respectively. ## task and dataset descriptions the goal of this task is sentiment analysis in tamil and tulu, explicitly focusing on determining sentiment polarity in social media comments. this task aimed to develop two systems that can individually identify sentiment polarity from a given set of texts in tamil or tulu. to achieve this, we utilized the corpora provided by the shared task organizers foot_0 for sentiment analysis in tamil @xcite and tulu @xcite . the task required classifying texts into four predefined classes, positive, negative, mixed feeling, and neutral/unknown state, for tamil and tulu code-mixed texts. ## methodology this section summarized the methods and techniques applied for sentiment analysis in tamil and tulu. figure 1 outlines the employed techniques for sa in tamil and tulu. ## results and analysis for the tulu test set, as shown in ## conclusion this paper investigated different ml, dl, and transformer-based models for categorizing textual sentiment in tamil and tulu. the experimental results revealed that the xlm-roberta model obtained the highest macro f1 score (0.258) for the tamil textual comments, whereas the m-bert model scored the best (0.468) for the tulu texts. for future research, collaborative efforts are necessary to create resources, especially for low-resource languages like tamil and tulu, and add more diverse data to the existing corpora. additionally, improving the performance of sentiment analysis models for these languages will depend heavily on developments in pre-training methods and the availability of more thorough language representations.