H ausa NLP at S em E val-2023 Task 12: Leveraging A frican Low Resource T weet D ata for Sentiment Analysis (2023)

Abstract: We present the findings of SemEval-2023 Task 12, a shared task on sentiment analysis for low-resource African languages using Twitter dataset. The task featured three subtasks; subtask A is monolingual sentiment classification with 12 tracks which are all monolingual languages, subtask B is multilingual sentiment classification using the tracks in subtask A and subtask C is a zero-shot sentiment classification. We present the results and findings of subtask A, subtask B and subtask C. We also release the code on github. Our goal is to leverage low-resource tweet data using pre-trained Afro-xlmr-large, AfriBERTa-Large, Bert-base-arabic-camelbert-da-sentiment (Arabic-camelbert), Multilingual-BERT (mBERT) and BERT models for sentiment analysis of 14 African languages. The datasets for these subtasks consists of a gold standard multi-class labeled Twitter datasets from these languages. Our results demonstrate that Afro-xlmr-large model performed better compared to the other models in most of the languages datasets. Similarly, Nigerian languages: Hausa, Igbo, and Yoruba achieved better performance compared to other languages and this can be attributed to the higher volume of data present in the languages.

## introduction social media offers an opinionated platform of data content on many topics of interest, such as product reviews, feedback on purchases or services, political interest, etc., that is dynamically created by users in different languages. sentiment analysis in many languages is required due to the need for effective classification of these contents. however, the majority of research on sentiment analysis has been done in high-resource languages @xcite @xcite @xcite while several lowresource african languages receive little attention in natural language processing (nlp) application due to insufficient of publicly available data. although, recently, some considerable efforts are made for the development of sentiment analysis for some low-resource african languages @xcite @xcite @xcite , nonetheless, sentiment analysis for lowresourced african languages still is a misrepresented research area. the afrisenti foot_0 shared task 12 @xcite in this paper, we demonstrate our approach to tackling the concerns raised in semeval task 12 as well as the obstacles posed by low-resource languages @xcite . we used the bert, afriberta_large and afro-xlmr-large model to facilitate the classification of tweets using a monolingual, multilingual and zero shot approach. the rest of the paper is organized as follows. section 2 is the related works. section 3 describes the proposed approach. experimentation and evaluation are discussed in section 4, while section 5 draws some conclusions and discusses some directions for future work. ## related work sentiment analysis is the process of generating positive or negative sentiment from data using computational techniques, and to some extent is capable of predicting and classifying tempers such as excitement, anger, and sadness @xcite . sentiment and emotion are considered to be related, @xcite therefore, the process of assigning polarity to text and emotion among others has become a prevalent task in nlp. @xcite . hence, sentiment analysis is to generate opinions based on a given input provided by users @xcite , the current campaign in nlp is that sentiment analysis is extensively adopted for opinion mining to collect information about users from different fields of a particular aspect. although sentiment analysis has been extensively used in many high-resource languages like english and french just to mention a few, little attention is paid to african low-resource languages. @xcite presented the first extensive human-annotated twitter sentiment dataset for the hausa, igbo, nigerian-pidgin, and yoruba languages-the four most widely spoken in nigeria-consisting of roughly 30,000 annotated tweets per language (and 14,000 for nigerian-pidgin) and a sizable portion of codemixed tweets. for these low-resource languages, they suggested text collecting, filtering, processing, and labeling techniques that let us build datasets. they used a variety of pre-trained models and transfer methods. they discovered that the most effective methods are language-specific models and language-adaptive fine-tuning. ## shared task description the afrisenti-semeval shared task 12 is based on a collection of twitter datasets in 14 african languages for sentiment classification. participants are provided with a training dataset and are required to make a prediction using multiclass sentiment classification. it consists of three sub-tasks: monolingual sentiment classification, multilingual sentiment classification, and zero-shot sentiment classification. in this paper, we concentrate on all the three subtasks with a total of 14 languages. ## proposed approach in this section, we describe our proposed approach for the semeval shared task i.e leveraging low-resource tweet data for sentiment analysis of african languages. our goal is to identify the sentiment classification of low-resource african languages using afriberta-large, bert-basemultilingual-cased and afro-xlmr-large models. we have also shared the code on github foot_1 . ## experiment and evaluation this section describes the experimental and evaluation settings for our proposed approach for the semeval-2023 task 12. ## results and discussion for the monolingual subtask, the best dataset performance across the pre-trained models is the hausa language with an average of 74.50%, followed by igbo and yoruba with 71.32% and 64.55.% respectively. in terms of model performance, each of the pre-trained model performed best in at least one of the monolingual dataset. however, afro-xlmr-large ranked highest in best performances in 7 monolingual datasets, followed by afriberta-large and arabic-camelbert with 3 and 2 best performances respectively. bert and mbert with least best performances of 1 each. for multilingual subtask, afro-xlmr-large also performed best. this is much expected since the dataset is composed of all the 12 languages in the monolingual subtask. the impressive 69.50% performance is also attributed to the large volume of dataset. afriberta-large performance was also impressive with 69.30% just 0.20% difference. this performance shows that there's a potential for building a cross-lingual model for a more advanced nlp system. for zero shot subtask, with two datsets, afriberta-large and afro-xlmr-large share the top spots with each emerging best in 1 of the tracks. afriberta-large dominated the tigrinya zero shot track with an impressive 62.50% performance. while afro-xlmr-large still claim another excellent performance on oromo dataset track with 46.20% as the best score. lastly, to average the performance across all the three subtasks, afro-xlmr-large came top with overall top score of 62.26%. it was trailed behind by afriberta-large with 60.53%. mbert, arabic-camelbert and bert coming distant third with 59.53.00%, fourth with 56.35% and fifth with 53.12% respectively. ## conclusion and future work in this paper, we presented our system description for the semeval shared task on sentiment analysis for low-resource african languages using twitter dataset. the task consists of three sub-tasks: monolingual, multilingual, and zero-shot. several pretrained llms were used for fine-tuning. afro-xlmr-large performed relatively best across the three subtasks coming top in 9 out of 15. afriberta came top in 4, while bert-base-arabiccamelbert-da-sentiment performed best in arabic datasets of darija and algerian arabic. bert and mbert also manage to came top in 1 task each. experimental results demonstrated that nigerian languages: hausa, igbo and yoruba achieved better performance compared to other languages due to the higher volume of data present in the languages. our results indicate that deep learning are effective in sentiment classification in the african language given the right data, model and training. for future work, to incorporate other data sources such as moview reviews or news articles for generalizability. we also recommend fine tuning for specific individual languages to incorporate linguistic features specific to each language, such as idioms or colloquialisms.