GSAC : A G ujarati Sentiment Analysis Corpus from T witter (2023)

Abstract: Sentiment Analysis is an important task for analysing online content across languages for tasks such as content moderation and opinion mining. Though a significant amount of resources are available for Sentiment Analysis in several Indian languages, there do not exist any large-scale, open-access corpora for Gujarati. Our paper presents and describes the Gujarati Sentiment Analysis Corpus (GSAC), which has been sourced from Twitter and manually annotated by native speakers of the language. We describe in detail our collection and annotation processes and conduct extensive experiments on our corpus to provide reliable baselines for future work using our dataset.

## introduction sentiment analysis is an nlp task that involves identifying the sentiment or the polarity (such as positive vs negative) of a piece of text @xcite . it has received significant attention in recent years, with ever-increasing internet access and social media usage, even in indian languages such as hindi @xcite and marathi @xcite which are typologically and geographically close to gujarati. however, there is hardly any work done in gujarati itself. gujarati is a very prominent language in western india, with over 55 million first-language speakers and a significant presence in the states of gujarat, maharashtra, and rajasthan @xcite . it is also the official language of the state of gujarat. despite a large online community active on social media and a significant mainstream media presence, there is a lack of large-scale, publicly available resources for sentiment classification (see section 2). hence, we describe a new, gold-standard, manually annotated gujarati sentiment analysis corpus (gsac) for monolingual sentiment classification. the dataset is sourced from twitter and labelled by native speakers. we describe our annotation process and also run extensive experiments on the dataset using feature-based and deep-learning architectures to establish a reliable baseline for gsac and compare the performances of various model architectures. the dataset is available on github. 1 2 related work significant work has been done on coarse-grained and aspect-based sentiment analysis (sa) in various indian languages. datasets have been created for sa in hindi @xcite , @xcite , marathi @xcite , @xcite and tamil @xcite , and tamil and malyalam @xcite . however, sa in gujarati has been scarcely explored, and no standard, publicly available dataset exists. one of the earliest works in sa in gujarati was by @xcite , who used a pos tag-based feature set for an svm classifier on a small sample of 40 tweets. since then, @xcite developed and experimented with a gujarati sentiwordnet to classify tweets, creating a twitter dataset with 1120 samples. other approaches included scraping movie-review websites to create a dataset @xcite , even translating reviews from english to gujarati to expand the dataset @xcite . @xcite attempted classifying a set of 300 poems into nine different emotional categories using machine learning-based approaches. however, none of the datasets used in these experiments have been released to open access, which makes it difficult to reproduce any of these results or compare the performance of new models with work(s) source size annotation open access @xcite twitter 40 manual no @xcite poems 300 manual no @xcite twitter 1120 manual no @xcite , @xcite movie reviews 500 manual no @xcite , @xcite movie reviews (gujarati + translated from english) 2085 automated, based on website rating no ## dataset creation the dataset was created in two main steps -collecting and sampling the dataset from twitter to create a subset for annotation and getting the data annotated by native speakers, which included creating the annotation guidelines and training them for the task. ## experiments we train two sets of models to test how different models perform on our dataset and to set baselines for it. the first set of models consists of feature vector-based models, which we train on two different variants based on different sets of features -bag-of-words and tf-idf. the second set is a set of deep contextualised models, where we fine-tune various transformer-based pre-trained language models for classification on this dataset. ## results we report the detailed results for each model in ## conclusion and future work in this paper, we present the gujarati sentiment analysis corpus (gsac), which contains over 6500 manually annotated tweets. to the best of our knowledge, it is the first significant publicly available corpus for this task in gujarati. we also present our annotation schema and conduct extensive experimentation to establish baselines for this new dataset. we find that pre-trained language models that included gujarati as a part of pretraining or fine-tuning achieve better performance on this dataset compared to other models, with in-dicbert achieving the best weighted and macro f1 scores. as a part of future work, we plan to explore methods to extend this dataset automatically by using this dataset as a seed dataset to label additional data (such as by bootstrapping) or by exploring other avenues of acquiring data, such as via machine translation of existing datasets in other languages such as english or hindi.