Sentiment Analysis for K onkani using Zero-Shot M arathi Trained Neural Network Model (2024)

Abstract: Sentiment Analysis plays a crucial role in understanding user opinions in various languages. The paper presents an experiment with a sentiment analysis model fine-tuned on Marathi sentences to classify sentiments into positive, negative, and neutral categories. The fine-tuned model shows high accuracy when tested on Konkani sentences, despite not being explicitly trained on Konkani data; since Marathi is a language very close to Konkani. This outcome highlights the effectiveness of Zero-shot learning, where the model generalizes well across linguistically similar languages. Evaluation metrics such as accuracy, balanced accuracy, negative accuracy, neutral accuracy, positive accuracy and confusion matrix scores were used to assess the performance, with Konkani sentences demonstrating superior results. These findings indicate that zero-shot sentiment analysis can be a powerful tool for sentiment classification in resource poor languages like Konkani, where labeled data is limited. The method can be used to generate datasets for resource-poor languages. Furthermore, this suggests that leveraging linguistically similar languages can help generate datasets for low-resource languages, enhancing sentiment analysis capabilities where labeled data is scarce. By utilizing related languages, zero-shot models can achieve meaningful performance without the need for extensive labeled data for the target language.

## introduction sentiment analysis @xcite , the computational task of identifying and categorizing opinions expressed in text, has gained significant traction in recent years due to the proliferation of online content, particularly product reviews and social media interactions. understanding sentiments is vital for businesses and public services aiming to tailor their products and services to meet customer expectations and enhance user experience. while various sentiment analysis models have been developed for languages with abundant resources, many low-resource languages, including konkani, remain underserved in this area. konkani (mascarenhas-keyes, 2020; @xcite is a language primarily spoken along the western coast of india and has its own unique linguistic features. despite its rich cultural heritage, the lack of digital resources and language technology tools has hindered the development of effective natural language processing (nlp) applications for konkani. on the other hand, marathi @xcite , a language closely related to konkani, has seen advancements in sentiment analysis due to better resource availability and research focus. in this study, we aim to investigate the applicability of a pre-existing marathi sentiment analysis models to konkani text. by examining the performance of this model on both marathi and konkani datasets, we seek to highlight the discrepancies in accuracy and effectiveness, ultimately demonstrating the necessity for a dedicated konkani sentiment analysis datasest and models. our research underscores the importance of developing languagespecific tools that can cater to the unique characteristics of low-resource languages, thereby improving sentiment analysis capabilities in diverse linguistic contexts. the remainder of this paper is structured as follows: section 2 reviews related work on sentiment analysis in low-resource languages and crosslingual transfer learning. section 3 describes the methodology, including the dataset used, the model architecture, and the training process. section 4 presents the experimental details, comparing the performance of the model before and after finetuning. section 5 discusses the implications of the results and the potential for further improvements in accuracy. finally, section 6 concludes the paper, highlighting that the creation of a konkani-specific labeled dataset and further fine-tuning could significantly increase the model's performance in sentiment classification. ## literature survey zero-shot learning (zsl) is a paradigm in natural language processing (nlp) that allows models to perform tasks without prior exposure to specific examples or labeled data. this capability is particularly valuable in sentiment analysis, where traditional models often require extensive training on labeled datasets. zsl leverages a model's innate linguistic knowledge to generalize across tasks it has not encountered during training, making it adaptable to new languages and contexts @xcite . recent advancements in pre-training models like bert and its multilingual variants have enhanced the zero-shot performance by capturing cross-linguistic knowledge, thus allowing the models to generalize well to unseen languages and domains @xcite . zero-shot classification models can categorize text into various sentiments (positive, negative, neutral) based on their understanding of language relationships rather than relying on task-specific training data. this approach accommodates a broader range of emotional expressions. for instance, models can be prompted to classify sentiments related to mental health by providing specific emotional labels such as joy, sadness, or anxiety. a similar approach is applied in cross-lingual settings, where pre-trained models like multilingual bert can transfer knowledge across languages, achieving high performance in tasks like sentiment analysis without task-specific annotations in the target language @xcite . recent studies have demonstrated that zero-shot sentiment analysis can effectively extend to lowresource languages. by pre-training models using multilingual lexicons or using libraries such as indicnlp @xcite , researchers have achieved superior performance in sentiment classification across languages like marathi and konkani. this method reduces the need for extensive sentence-level annotations and allows for effective cross-linguistic transfer @xcite . such models, when pre-trained on diverse language corpora, can generalize across various languages, making them particularly useful for low-resource languages that lack large annotated datasets @xcite . despite its advantages, zero-shot learning faces several challenges. models may struggle with tasks requiring specialized knowledge or intricate understanding of context, such as sarcasm or culturally embedded sentiments. additionally, zero-shot models might exhibit limitations in grasping the full context of sentiment-laden statements, particularly when subtlety is involved @xcite . the effectiveness of zero-shot learning heavily relies on the quality and breadth of the pre-training data, as models trained on general language patterns may not perform well in highly specialized domains. one innovative approach involves using multilingual sentiment lexicons for pre-training models, which has shown promising results in improving zero-shot performance for low-resource languages @xcite . the performance of zero-shot sentiment analysis models is typically assessed using various metrics such as accuracy, balanced accuracy, and confusion matrix scores, which help in evaluating their generalizability across different languages and domains. the exploration of zero-shot sentiment analysis in low-resource languages like marathi and konkani highlights its potential as a transformative tool in nlp. by leveraging general linguistic knowledge and multilingual resources, researchers can effectively analyze sentiments without extensive labeled data. however, ongoing research is necessary to address the challenges associated with domain specificity and contextual understanding @xcite . ## methodology we use two pre-trained models on marathi as base models to perform the sentiment analysis on konkani and marathi datasets. 1. deepampatel/roberta-mlm-marathi @xcite 2. l3cube-pune/marathisentiment (l3cube-pune, 2024) ## results and discussions this study demonstrates the significant impact of fine-tuning on sentiment classification performance for both marathi and konkani. after fine-tuning, both models-deepampatel/roberta-mlm-marathi and l3cube-pune/marathisentiment-showed improved accuracy in classifying marathi sentiments, particularly for deepampatel/roberta-mlm-marathi. additionally, both models displayed even better performance when applied to konkani sentences in a zero-shot learning setting, with high accuracy in detecting positive sentiments. these results highlight the effectiveness of cross-lingual transfer learning between linguistically similar languages, such as marathi and konkani. the l3cubepune/marathisentiment model particularly benefited from fine-tuning, as it showed stronger results for negative sentiments in konkani. this suggests that, while fine-tuning on marathi improves sentiment classification, incorporating konkani-specific data in future experiments would further enhance accuracy across all sentiment classes, thereby contributing to more reliable nlp applications for lowresource languages. the experiment was an attempt to show that although resource poor languages do not have datasets, they can be created with the help of similar languages. the two models considered are as follows, model 1: deepampatel/roberta-mlm-marathi and model 2: l3cube-pune/marathisentiment. ## conclusion based on the results presented, we observed that fine-tuning the models significantly improved sentiment classification accuracy for both marathi and konkani. model 1 demonstrated better performance after fine-tuning, particularly in the zeroshot application to konkani, where the positive sentiment classification was especially strong. model 2 also performed well in the zero-shot learning scenario. the fine-tuned models showed better performance in sentiment classification across both languages and we expect that additional training on konkani-specific data would further improve performance. our study highlights that the method used in this research can be employed to create sentimenttagged data for konkani despite the absence of large, annotated datasets. the cross-lingual transfer from marathi to konkani proves effective, making this approach a technique we look forward to for building sentiment-tagged datasets for lowresource languages like konkani, where labeled data is limited.