P ing A n L ife I nsurance at S em E val-2023 Task 12: Sentiment Analysis for Low-resource A frican Languages with Multi-Model Fusion (2023)

Abstract: This paper describes our system used in the SemEval-2023 Task12: Sentiment Analysis for Low-resource African Languages using Twit- ter Dataset (Muhammad et al., 2023c). The AfriSenti-SemEval Shared Task 12 is based on a collection of Twitter datasets in 14 African languages for sentiment classification. It con- sists of three sub-tasks. Task A is a monolin- gual sentiment classification which covered 12 African languages. Task B is a multilingual sen- timent classification which combined training data from Task A (12 African languages). Task C is a zero-shot sentiment classification. We uti- lized various strategies, including monolingual training, multilingual mixed training, and trans- lation technology, and proposed a weighted vot- ing method that combined the results of differ- ent strategies. Substantially, in the monolingual subtask, our system achieved Top-1 in two lan- guages (Yoruba and Twi) and Top-2 in four languages (Nigerian Pidgin, Algerian Arabic, and Swahili, Multilingual). In the multilingual subtask, Our system achived Top-2 in publish leaderBoard.

## introduction sentiment analysis (sa) refers to the identification of sentiment and opinion contained in the inpfut texts that are often user-generated comments. the problem of automatic sentiment analysis is growing research topic. although sentiment analysis is an important area and already has a wide range of applications, it clearly is not a straightforward task and has many challenges related to natural language processing (nlp). in recent years, sentiment analysis has gained significant attention due to its numerous vital applications. the growth of social networks has resulted in the generation of more complex and interrelated information, enabling various artificial intelligence (ai) applications such as sentiment analysis, machine translation, and detection of harmful content. social media platforms have become impor-tant sources of data for sentiment analysis. however, most sentiment analysis research focuses on high-resource languages such as english, while low-resource languages like african languages remain underrepresented. according to @xcite , around 2,058, or 30%, of all living languages are african languages. however, most of these languages lack curated datasets required for the development of ai applications, and this issue is not unique to sentiment analysis, but affects nlp research as a whole. sentiment analysis has garnered increasing interest due to its applicability across many domains, including public health, commerce/business, art and literature, social sciences, neuroscience, and psychology. previous shared sentiment analysis tasks have been conducted, such as those by @xcite , @xcite , @xcite , and @xcite , among others. however, none of these tasks have included african languages, highlighting the need for concerted efforts to create resources for such languages @xcite . in this paper, we describe a system for sentiment analysis tasks in african low-resource languages on the twitter dataset. specifically, we introduce a novel ensemble system. for different languages, different strategies are proved effective in our system. to conduct monolingual training on each language data, we initially employed several pre-training models such as naija-twitter-sentimentafriberta-large 1 , twhin-bert 2 , and deberta 3 model. we also utilized multilingual mixed training by using four folds of each language as the training dataset and one fold of each language as the validation dataset, and implemented 5-fold cross-validation during training. additionally, we used translation technology to translate all languages into english and used both single language training and multilingual mixed training methods. finally, we combined the results of each approach through weighted voting to obtain the ultimate outcome. ## background sentiment analysis is the automated process of understanding and recognizing human sentiment from text, audio, or video. it uses natural language processing, text mining, and machine learning techniques to identify, extract, quantify, and study sentiment states from text. it can be used to detect and categorize positive, negative, and neutral sentiment in a given dataset. some subcategories of research in sentiment analysis include: multimodal sentiment analysis, aspect-based sentiment analysis, fine-grained opinion analysis, language specific sentiment analysis. traditional methods such as lexicon-based approaches @xcite , feature-based models @xcite and rule-based models @xcite have been applied to sentiment analysis tasks. more recent approaches have focused on using machine learning algorithms to improve the accuracy and efficiency of sentiment analysis @xcite . in particular, deep learning methods are being applied to a wide range of natural language processing tasks, including sentiment analysis @xcite . these methods have been proven to be highly effective in extracting sentiment from text. additionally, there has been research into utilizing transfer learning and ensemble methods for sentiment analysis tasks. recently, pre-training methods have shown their powerfulness in learning general semantic representations, and have remarkably improved most natural language processing (nlp) tasks like sentiment analysis. these methods build unsupervised objectives at word-level, such as masking strategy, next-word prediction or permutation. such word prediction based objectives have shown great abilities to capture dependency between words and syntactic structures. however, due to the most of pre-training models only pre-trained in highresource languages such as english, the african low-resource languages unable to perform semantic analysis well. ## system overview we utilized various approaches in this work, incorporating weighted voting to combine outcomes from different strategies. initially, we employed several pre-training models, including naija-twittersentiment-afriberta-large, twhin-bert and de-berta model, to conduct monolingual training on each language data. additionally, we utilized multilingual mixed training to perform joint training, using four folds of each language as the training dataset and one fold of each language as the validation dataset, and implementing 5-fold crossvalidation during training. furthermore, we leveraged translation technology to translate all languages into english, and used both the single language training and multilingual mixed training methods for training. finally, we combined the outcomes of each approach through weighted voting to obtain the ultimate result. the model is capable of classifying tweets into three sentiment categories: negative, neutral, and positive. specifically, the naija-twitter-sentiment-afriberta-large model is an xlm-roberta-large model that was fine-tuned using an aggregate of four nigerian language datasets obtained from the naijasenti dataset. ## results in this section, the results of the experiments for our runs will be discussed and compared to the results published in through the experiment, we found the following conclusions: ## conclusion in this paper, we proposed a system to sentiment analysis for low-resource african languages using twitter dataset. to train the model, we tried different pre-trained models as the base model and used both monolingual language and multilingual mixed training techniques. we also used translation technology to convert all languages into english and used deberta as the base model for training. ultimately, we merged the results of multiple strategies using a weighted voting method. the system performs well in afrisenti-semeval, obtained two top-1 and three top-2 in task a, achieved the top-2 in task b. moving forward, we will continue to validate the effectiveness of more base models in different languages, and at the same time attempt to translate all languages into other african languages (such as hausa, igbo, etc.).