UBC - DLNLP at S em E val-2023 Task 12: Impact of Transfer Learning on A frican Sentiment Analysis (2023)

Abstract: We describe our contribution to the SemEVAl 2023 AfriSenti-SemEval shared task, where we tackle the task of sentiment analysis in 14 different African languages. We develop both monolingual and multilingual models under a full supervised setting (subtasks A and B). We also develop models for the zero-shot setting (subtask C). Our approach involves experimenting with transfer learning using six language models, including further pretraining of some of these models as well as a final finetuning stage. Our best performing models achieve an F1-score of 70.36 on development data and an F1-score of 66.13 on test data. Unsurprisingly, our results demonstrate the effectiveness of transfer learning and finetuning techniques for sentiment analysis across multiple languages. Our approach can be applied to other sentiment analysis tasks in different languages and domains.

## introduction sentiment analysis, also referred to as opinion mining, is a natural language processing (nlp) technique that aims to identify, extract, and evaluate opinions, attitudes, perceptions, and sentiments towards topics, products, services, and individuals from textual data @xcite . with the increased accessibility of the internet, people are increasingly sharing their opinions on various platforms, such as forums, blogs, wikis, websites, and social media pages. consequently, there is a need for the automatic extraction of sentiments to gain valuable insights into user perception, popular opinion, and trends @xcite . despite the increasing popularity of sentiment analysis, its application in low-resource african languages is still under-explored @xcite . this is because many african languages have limited digital resources, such as annotated data and lexical resources, which can hinder the development and evaluation of sentiment analysis models. so far, only a handful of african languages have few datasets for sentiment analysis @xcite @xcite @xcite . furthermore, african languages often exhibit complex morphology, syntax, semantics, stylistics, pragmatic and orthographic conventions including the use of diacritics, and code-mixing that can make it difficult to accurately identify and extract sentiment from text data @xcite . for instance, for some african languages, a single change in tone assignment can change the sentiment of a text @xcite . in this task, we conduct sentiment analysis on 14 african languages including algerian arabic, amharic, hausa, igbo, kinyarwanda, moroccan arabic, mozambican portuguese, nigerian pidgin, oromo, swahili, tigrinya, twi, and yoruba. the sentiment analysis data used for this shared task is the largest and most multilingual dataset for sentiment analysis for african languages to date @xcite . our contribution is as follows: the rest of this paper is organized as follows: we discuss existing literature in section 2, and provide background information in section 3. section 4 has details about the models we develop. in section 5 we describe each experiment performed and show the results on dev. and test sets in section 6. we conclude in section 7. ## approach we perform sentiment analysis on three different subtasks with 14 languages spoken across africa. the languages are quite diverse belonging to four different language families and written in different scripts including arabic, ethiopic, and latin scripts. we provide details about the languages and the datasets. ## system overview in order to identify the best-performing model for our datasets, we first finetuned 6 lms on the data from sub tasks a and b. specifically, we finetuned mbert, xlm-r, afro-xlmr, afriberta, afriteva, and serengeti. we also further pretrained afro-xlmr and serengeti. we refer to the pre-trained models as afro-xlmr-lm and serengeti-lm, respectively. we provide further details for each of the lms in what follows. ## experimental setup all our models are implemented using the py-torch framework and the open-source huggingface transformers libraries. all the models were trained on a single nvidia a100. all our models are trained using adam optimizer with a linear learning rate scheduler. after hyperparameter tuning using optuna, it was found that the optimum learning rate, batch size and number of epochs is 5 * e -5, 16 and 50 respectively. for the focal loss, the hyperparameters γ and α are set to 2 and 0.8, respectively. all models are evaluated on the weighted f 1 metric which was also used the objective for fine-tuning. for further pretraining of serengeti and afro-xlmr we used a more aggressive learning rate of 4 * e 4 using a batch size of 16 for 75 epochs. ## results we show the results on the dev. set for each model in ## conclusion we reported our participation in the three substacks for the afrisenti-semeval 2023 shared task. we described our transfer learning approaches using finetuning and further pretraining of existing lms. we show the performance of our models across the 14 languages in the three subtask.