Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon (2024)

Abstract: Improving multilingual language models capabilities in low-resource languages is generally difficult due to the scarcity of large-scale data in those languages. In this paper, we relax the reliance on texts in low-resource languages by using multilingual lexicons in pretraining to enhance multilingual capabilities. Specifically, we focus on zero-shot sentiment analysis tasks across 34 languages, including 6 high/medium-resource languages, 25 low-resource languages, and 3 code-switching datasets. We demonstrate that pretraining using multilingual lexicons, without using any sentence-level sentiment data, achieves superior zero-shot performance compared to models fine-tuned on English sentiment datasets, and large language models like GPT–3.5, BLOOMZ, and XGLM. These findings are observable for unseen low-resource languages to code-mixed scenarios involving high-resource languages.

## introduction when it comes to under-represented languages, multilingual language models @xcite @xcite are often considered the most viable option in the current era of pretraining and finetuning, primarily due to the scarcity of labeled and unlabeled training data. however, the limited language coverage of these models often results in poor cross-lingual transfer to under-represented languages @xcite . prior work has extended multilingual models @xcite to other languages by language-adaptive pretraining (i.e., continuing to pretrain on monolingual text) (e.g., @xcite and leveraging adapters @xcite . however, these language adaptation techniques are not 1 code and dataset can be found at: https://github. com/fajri91/zeroshotmultilingualsentiment compatible with low-resource languages due to the unavailability of adequate unlabeled monolingual texts. lexicons are more readily accessible and offer broader language coverage than monolingual corpora like wikipedia and the bible, making them a promising resource for extending multilingual models to under-represented languages. this is because when studying a new language, a lexicon is generally the first resource that field linguists develop to document its morpho-phonemics and basic vocabulary. of the 7,000+ languages spoken worldwide, lexicons are available for approximately 70% of them, while mbert, wikipedia/commoncrawl, and the bible are available for only 1%, 4%, and 23%, respectively @xcite . in prior work, @xcite proposed to use the panlex translation lexicon (baldwin et al., 2010), foot_0 to extend the language coverage of multilingual bert (mbert: @xcite ). they further pretrained mbert using synthetic texts generated through word-to-word translation, resulting in improvements in named-entity recognition tasks. drawing inspiration from their work, our study aims to reassess the utility of sentiment lexicons in sentiment analysis tasks, which were once a prominent feature in sentiment analysis prior to the advent of pre-trained language models. specifically, we seek to answer the following questions: we chose sentiment classification as the focus of our study for two reasons. first, there is a wealth of sentiment classification datasets across diverse languages, allowing us to conduct experiments across 34 languages, including 6 high/medium-resource languages, 25 low-resource languages, and 3 codeswitching language pairs. secondly, compared to other semantic tasks such as hate speech detection @xcite and emotion recognition (abdul-mageed and ungar, 2017; sosea and caragea, 2020), sentiment lexicons have been studied extensively and are wellestablished in the field. cross-lingual transfer in sentiment classification is a prime case of low-resource nlp. however, existing research has predominantly focused on high/medium-resource languages @xcite @xcite , relying on sentence-level sentiment datasets in english. in this paper, we showcase how models trained on english datasets are suboptimal for low-resource languages, and introduce lexicon-based pretraining that improves multilingual sentiment modeling. our contributions can be summarized as follows: ## related work we briefly review three subtopics that are pertinent to this work: (1) sentiment lexicons, (2) crosslingual adaptation for sentiment analysis, and (3) sentiment analysis in low-resource languages. sentiment lexicons a sentiment lexicon is a curated collection of words and phrases that are classified as bearing positive or negative polarity. such lexicons have applications in fields including nlp, cognitive science, psychology, and social science @xcite . there are two broad approaches to creating a sentiment lexicon: (1) direct annotation @xcite -have annotators assign sentiment scores to individual words on a rating scale, typically ranging from -5 (indicating very negative) to +5 (indicating very positive), or based on positive/negative categorical labels @xcite ; and (2) best-worst scaling (bws: @xcite ; mohammad (2018)) -have annotators select the most positive and least positive word from a collection of n words, and infer a sentiment score based on the global rank of all words in the collection. bws is considered more reliable than direct annotation as it helps mitigate annotator bias when assigning sentiment scores to individual words. these sentiment lexicons have been constructed predominantly for english, but they also exist for languages such as indonesian @xcite , arabic @xcite , persian @xcite , dutch @xcite , and spanish @xcite . in this work, we use nrc-vad @xcite , which is the largest english lexicon (19,965 words) and was built using the bws method. existing non-english lexicons are not just limited in size, they were also generally curated using a less reliable method (i.e., direct annotation). we instead use multilingual nrc-vad lexicons in 108 languages, which is created by the original authors of nrc-vad via google translate. 3 ## analysis regression vs. classification in lexicon-based pretraining in performance over unseen low-resource languages we compute the average results for languages that are completely unseen by all models, including 7 nusax languages (ace, ban, bbc, bjn, bug, mad, nij) and 4 african languages (dz, ma, pcm, pt-mz, aeb). we exclude lexicon-based pretraining with the panlex extension since its performance is poor for low-resource languages. as a comparison, we include code-switched text as seen languages for the models. figure 3 presents the performance of xlm-r large and mt5 large , which outperform the llms and models fine-tuned on sst. this suggests that the multilingual sentiment lexicon is effective at enhancing language generalization for low-resource languages. ## discussion given the positive results, we explore the potential applicability of our methodology to nlp tasks beyond sentiment analysis, offering valuable directions for future research. we examine if the lexicon-based pretraining yields benefits in other semantic tasks, including stance detection @xcite , hate speech detection @xcite , and emotion classification @xcite . for each task, we take two datasets and perform experiments in few-shot training using mbert base , following the setup described in section 4.2. instead of using the multilingual lexicon, we use the english nrc-vad lexicon since all the data is in english. for detailed information about the datasets and results, see the appendix. ## conclusion we have demonstrated the efficacy of employing a multilingual sentiment lexicon for achieving multilingual generalization in language model pretraining. without utilizing sentence-level datasets in any language, we provide compelling evidence of superior zero-shot performance in sentiment analysis tasks for low-resource languages, surpassing the performance of large language models. these findings open up new avenues for research in the realm of low-resource languages, not only for language understanding but also language generation tasks. our results encourage further exploration and investigation of this exciting research direction.