S eals_ L ab at S em E val-2023 Task 12: Sentiment Analysis for Low-resource A frican Languages, H ausa and I gbo (2023)

Abstract: One of the most extensively researched applications in natural language processing (NLP) is sentiment analysis. While the majority of the study focuses on high-resource languages (e.g., English), this research will focus on low-resource African languages namely Igbo and Hausa. The annotated tweets of both languages have a significant number of code-mixed tweets. The curated datasets necessary to build complex AI applications are not available for the majority of African languages. To optimize the use of such datasets, research is needed to determine the viability of present NLP procedures as well as the development of novel techniques. This paper outlines our efforts to develop a sentiment analysis (for positive and negative as well as neutral) system for tweets from the Hausa, and Igbo languages. Sentiment analysis can computationally analyze and discover sentiments in a text or document. We worked on the first thorough compilation of AfriSenti-SemEval 2023 Shared Task 12 Twitter datasets that are human-annotated for the most widely spoken languages in Nigeria, such as Hausa and Igbo. Here we trained the modern pre-trained language model AfriBERTa large on the AfriSenti-SemEval Shared Task 12 Twitter dataset to create sentiment classification. In particular, the results demonstrate that our model trained on AfriSenti-SemEval Shared Task 12 datasets and produced with an F1 score of 80.85% for Hausa and 80.82% for Igbo languages on the sentiment analysis test. In AfriSenti-SemEval 2023 shared task 12 (Task A), we consistently ranked top 10 by achieving a mean F1 score of more than 80% for both the Hausa and Igbo languages.

## introduction natural language processing (nlp) is a method that allows computers to intelligently and effectively analyze, and comprehend meaning from human language @xcite . in nlp, trained language models have taken the highest position, displaying outstanding performance on a variety of nlp tasks @xcite because of their capacity to recognize and learn intricate linguistic correlations and patterns in natural language data. sentiment analysis (sa) @xcite is a method for computationally identifying and classifying opinions especially to determine if the author has positive, negative, or neutral emotions about a specific topic. furthermore, it has been shown that models like xlm-r @xcite and mbert @xcite generalize well to a variety of languages. these models are known to be challenging to utilize since training data is typically lacking for low-resource languages @xcite . it is difficult to adapt the majority of highresource language models successfully and approaches to low-resource african languages like hausa and igbo because of variations in linguistic, cultural, and technological contexts, particularly in social media communication. sa is a method for computationally identifying and classifying opinions especially to determine if the author has positive, negative, or neutral emotions about a specific topic. low-resource languages are not sufficiently supported by digital technology because there is a lack of training data for many nlp applications @xcite . as a result, developing resources for these languages calls for a deliberate effort. there are a few basic methods for sa on linguistic data @xcite . the purpose of afrisenti-semeval shared task 12 is to do sa on a collection of african language tweets for hausa and igbo. using the afrisenti-semeval shared task 12 (task a) twitter datasets, the afriberta large @xcite model was developed and trained for this task. our research relies mainly on the deep-learning technique, which entails creating a model by teaching the classifier using labeled samples from a positive, negative, or neutral class. first, take the features from the datasets and train the algorithm with them. the purpose of this research is to investigate the best ways to apply sentiment analysis (sa) to low-resource languages, with a focus on hausa and igbo. we also focus on how sentiment may be extracted from low-resource languages using modern deep-learning models and sa systems. ## related work sa has received greater attention for languages with high-resource bases like german and english @xcite . several studies have looked into creating a twitter corpus automatically or manually annotating one to do sentiment analysis on tweets @xcite . several perspectives have already been used to examine the problem of crosslingual sa @xcite , multilingual sentiment analysis @xcite , and monolingual sentiment analysis in a multilingual setting @xcite . sentence part-of-speech (pos) tagging in hausa was the first attempt at sentiment analysis in the language @xcite . however, it was also suggested that the pos of hausa sentences be assigned using the hidden markov model. due to the language's morphological, syntactic, and semantic complexity, natural language processing (nlp) tasks including sentiment analysis, named entity recognition (ner) @xcite , and part of speech tagging (pos) @xcite are difficult. amharic sentiment analysis is still challenging due to the absence of parsers and taggers in nlp tools and poorly annotated corpora @xcite . nlp problems have recently been significantly enhanced through unsupervised text representation learning @xcite . learning contextualized representations from pretrained word embeddings has improved static representations @xcite . this has been greatly enhanced by pretraining language models based on transformers @xcite . however, a significant fraction of this research has focused on high-resource languages that handle comparatively enormous amounts of data. although the almost 2000 languages spoken in africa account for 30.1% of all living languages, little focus has been placed on them. there have been a small number of studies on learning pretrained embeddings for african languages @xcite . the majority of them were static and trained on a single language @xcite . to date, only 3 of the 104 languages that mbert was pretrained on are african languages (small data). african languages make up 4.80 gb, or 0.2%, of the 2395 gb used to pretrained the xlm-r model @xcite . trained language models have been shown to perform effectively when there exists a lot of data @xcite . additionally, roberta language models trained on 10 to 100 million tokens can encode the majority of syntactic and semantic features @xcite in its learned text representations, while @xcite showed the state-of-the-art performance with elmo @xcite a language model pretrained on less than 1 gb of wikipedia text. these sa techniques are now widely employed in several industries, such as politics, education, tourism, entertainment, and business (mohammad, 2020). there are fewer resources available for sa in african languages than there are for languages like english and arabic. this is primarily due to a lack of funding and scholarly research in the area. numerous studies on nigerian codemixed english have been done already @xcite . @xcite built a twitter corpus for the hausa language and applied mixed hausa and english features to a classifier. a model was trained using texts that used the eight essential pos, as well as processes in the form of a number creator and a tense creator as independent pos tagged sets, on a corpus of hausa-based texts from freedom radio and afri hausa. @xcite examined igbo emotion words using interval type-2 fuzzy logic. using artificial intelligence to obtain the best level of accuracy improvement, the classifier can be adjusted to increase accuracy in the following work, according to the literature review. in other tasks at semeval, researchers have looked at the sa of product reviews and their aspects @xcite , sa of figurative language on twitter @xcite implicit event polarity, detecting stance in tweets (mohammad, 2020), and out-of-context sentiment intensity of words and phrases @xcite . although some of these researches incorporated languages other than english, such as arabic, they did not target tweets or focus on sentiment toward a topic @xcite . the main purpose of our studies is to perform sa on afrisenti-semeval 2023 shared task 12 twitter ( task a) datasets for the hausa and igbo languages. we employ deep-learning strategies to enhance classification outcomes. the primary goal of this effort is to train a multilingual language model for low-resource languages. the remainder of this paper is as follows. the background (task a) of afrisenti-semeval shared task 12 is detailed in section 3. the experimental setup of our process is described in section 4. in section 5, the system overview is discussed, along with the methods and resources. section 6 presents the results of the evaluation and section 7 ends by pointing to the conclusion and potential future research directions. ## background there are three sub-tasks in afrisenti-semeval shared task 12. our study concentrated on task a, monolingual sentiment classification, which classified sentiment (positive, negative, and neutral) using a collection of twitter datasets in 12 african languages @xcite . we used the semeval 2023 task 12 corpora to train multilingual language models in hausa and igbo (two widely spoken african languages) @xcite . in task a: in this sentiment classification, we used datasets in the target languages (hausa and igbo) to determine the sentiment of a tweet (positive, negative, and neutral). our top priorities were looking into low-resource language tools and testing deep-learning models for sentiment analysis in african languages. african languages are also low-resource languages due to the lack of readily available linguistic resources, corpora, and tools @xcite . ## results in this section, firstly, we compare various lowresourced language models based on training data to better understand how to train multilingual language models in limited datasets. we focused on lexical resources, tools, and features that are present in sentiment analysis systems, as well as the fundamental generic methods for sentiment analysis and pre-processing procedures. the test set for task a consisted of 5304 unlabeled tweets for hausa and 3683 unlabeled tweets for igbo. the method was described in the previous section. our system was ranked among the top 10 systems in the official evaluation results. with the fine-tuned afriberta, we discovered comparable f1 scores for afrisenti-semeval shared task 12 (task a) in both hausa and igbo. number of participating groups in task a (afrisenti-semeval shared task 12) was 35 teams for hausa and 33 teams for igbo languages. for all participants in afrisenti-semeval shared task 12 (task a), the results are shown in tables 7 and ta ble 8, where teams are ranked based on a weighted f1 score. this afrisenti-semeval task 12 is to categorize the texts as positive, negative, or neutral. the top-ranking team in hausa and igbo was bcai-air3. they obtained an 82.62% weighted f1 score for hausa and an 82.96% weighted f1 score for igbo languages. additionally, machamp teams received 17.02% weighted f1 score for hausa(ha) and 26.91% weighted f1 score for igbo languages. results show, in particular, that our team seals_ lab obtained an 80.85% weighted f1 ## conclusion sa is frequently seen as a straightforward classification task to divide tasks into positive, negative, and neutral. the task of sa, in comparison, is extremely difficult and influenced by a variety of factors, including human motivations, and intentions. however, these sa aspects remain either un or under-explored. afrisenti-semeval shared task 12 (task a) twitter sentiment analysis, attracted 35 teams for the hausa language and 33 teams for the igbo language this year. our submissions to this shared task (task a) for the hausa and igbo languages are summarised in this document and the final label for this task was determined using the deep-learning model afriberta large, which was designed for classifying the sentiment of twitter datasets. our system ranked top 10 in both (hausa and igbo )languages. although the outcomes from our model were comparable, further advancement is yet possible. in our future work, we plan to look at several topics that were not addressed in this research, such as the imbalance problem and the need to properly incorporate topic information into the model used for this task, and also need to take into account extra resources and extra features for twitter representation.