Investigating the Impact of Language-Adaptive Fine-Tuning on Sentiment Analysis in H ausa Language Using A fri BERT a (2025)

Abstract: Sentiment analysis (SA) plays a vital role in Natural Language Processing (NLP) by identifying sentiments expressed in text. Although significant advances have been made in SA for widely spoken languages, low-resource languages such as Hausa face unique challenges, primarily due to a lack of digital resources. This study investigates the effectiveness of Language-Adaptive Fine-Tuning (LAFT) to improve SA performance in Hausa. We first curate a diverse, unlabeled corpus to expand the model’s linguistic capabilities, followed by applying LAFT to adapt AfriBERTa specifically to the nuances of the Hausa language. The adapted model is then fine-tuned on the labeled NaijaSenti sentiment dataset to evaluate its performance. Our findings demonstrate that LAFT gives modest improvements, which may be attributed to the use of formal Hausa text rather than informal social media data. Nevertheless, the pre-trained AfriBERTa model significantly outperformed models not specifically trained on Hausa, highlighting the importance of using pre-trained models in low-resource contexts. This research emphasizes the necessity for diverse data sources to advance NLP applications for low-resource African languages. We will publish the code and the data set to encourage further research and facilitate reproducibility in low-resource NLP

## introduction sentiment analysis (sa) is a vital task in natural language processing (nlp) aimed at identifying and categorizing opinions expressed in text @xcite . although considerable progress has been made in this field, especially for widely spoken languages such as english @xcite , the same cannot be said for many low-resource languages, such as hausa @xcite . hausa is a chadic language spoken * correspondance to 2770930@students.wits.ac.za primarily by hausa people in the northern regions of nigeria, ghana, cameroon, benin and togo, as well as the southern areas of niger and chad, with notable minority communities in ivory coast @xcite @xcite . approximately 54 million people are estimated to speak it as their first language, while around 34 million use it as a second language, resulting in a total of about 88 million hausa speakers @xcite . it has limited digital resources, which present challenges for nlp research, including sa @xcite . recent advancements in pre-trained large language models (llms) have enabled the use of transfer learning to address challenges in nlp for low-resource languages. for example, multilingual models like bert (bidirectional encoder representations from transformers) have shown strong performance in various nlp tasks @xcite , but often struggle with low-resource languages due to limited data and linguistic diversity @xcite . language-adaptive fine-tuning (laft) has emerged as a promising approach to improve the handling of language-specific nuances in these models and improve performance in tasks such as sa, especially for underrepresented languages @xcite . in this study, we investigate the impact of laft on sa in hausa using pre-trained llm. we can summarize our main contributions as follows. ## related work language-adaptive fine-tuning (laft) has demonstrated its effectiveness in enhancing sentiment analysis (sa) performance in african languages @xcite . for example, fine-tuning multilingual pre-trained language models like afriberta on monolingual texts of african languages significantly improves sentiment classification tasks @xcite @xcite . afriberta, introduced by @xcite , represents a notable advancement in multilingual language modeling for african languages. it employs the transformer architecture, leveraging the standard masked language modeling (mlm) objective for pretraining. the model is available in two configurations: a small version with approximately 97 million parameters and a large version with around 126 million parameters. this flexibility allows it to cater to varying computational resource constraints while retaining its utility for african languages. pre-trained on 11 african languages, afrib-erta's training datasets were aggregated from bbc news websites and common crawl, totaling less than 1 gb of data and comprising 108.8 million tokens @xcite . although the dataset size is relatively small compared to those used for other popular language models, afriberta effectively captures the nuances of african languages, which is reflected in its performance on downstream nlp tasks @xcite . afriberta has been effectively utilized for sa in african languages such as hausa and igbo. in a study focusing on the afrisenti-semeval 2023 shared task 12, afriberta was trained on annotated twitter datasets for these languages. the model achieved impressive f1 scores of 80.85% for hausa and 80.82% for igbo, demonstrating its capability in handling sentiment classification tasks in low-resource languages @xcite . afriberta, when compared to other models like xlm-r @xcite and mbert @xcite , has shown competitive performance. for instance, in a multilingual adaptive fine-tuning approach, afriberta and xlm-r were evaluated on tasks including sentiment classification, and the results were comparable to individual language adaptations while requiring less disk space @xcite . another study highlighted that mbert outperformed other models like roberta and xlm-r in hausa sentiment analysis, achieving the highest accuracy and f1-score of 0.73% @xcite . however, afriberta's specialization for african languages provides a significant advantage in crosslingual transfer learning @xcite although multilingual fine-tuning can facilitate cross-lingual transfer learning, monolingual fine-tuning often gives superior results for specific languages. for instance, @xcite demonstrates that monolingual fine-tuning on datasets with thousands of samples produces optimal results. moreover, combining languageadaptive and task-adaptive pretraining on african texts, along with careful source language selection, can lead to remarkable performance improvements. this approach minimizes harmful interference from dissimilar languages and enhances outcomes in multilingual and cross-lingual contexts @xcite . systems utilizing laft have achieved high rankings in shared tasks, demonstrating substantial improvements in weighted f1 scores and other performance metrics @xcite . however, building reliable sa systems for lowresource african languages remains challenging due to the limited availability of training data @xcite . despite the promising results of laft and the benefits of monolingual fine-tuning, the scarcity of large high-quality datasets for low-resource african languages, such as hausa, poses a significant challenge. therefore, this study aims to contribute to the growing body of knowledge on sa for african languages by providing insights into the advantages of laft strategies in relation to hausa's linguistic characteristics and availability of data. ## results the results, averaged over three runs with a variation @xmath0.01, are presented across several metrics, comparing the model's performance before and after laft. a detailed analysis of both the baseline and laft models is provided below. ## discussion despite the subtle improvements in validation metrics, our findings align with previous studies by @xcite and @xcite , which demonstrate that fine-tuning a multilingual pre-trained language model (plm) on monolingual texts enhances sentiment classification performance for african languages. compared to previous sa works in hausa, our figure 5: attention map highlighting key phrases in sentiment analysis with strong focus on 'allah ya isa' indicating negative sentiment. on the right side, showing the model attending to the phrase "farin ciki" (glad) and "da" however, @xcite utilized the afriberta large model and achieved an accuracy of 81.2%. our findings with the smaller model (afriberta small) still show notable competitive performance, especially considering the model size. thus, our model performance not only validate the efficacy of the approach but also highlight the importance of using pre-trained models like afrib-erta that already incorporate african languages, leading to improved performance on sentiment classification tasks. ## conclusion in this study, we explored the use of laft for sa in hausa, a low-resource language, leveraging afriberta, which is pre-trained on african languages including hausa. afriberta's pre-training offered a notable advantage, outperforming models not trained on hausa by effectively capturing its linguistic nuances. although laft resulted in slight performance improvements, it did not significantly exceed the baseline set by afriberta's pretraining. this limited improvement is likely due to the fine-tuning corpus, which consisted mostly of formal text, contrasting with the conversational language commonly used in sentiment tasks. our results highlight the need for more diverse datasets that include informal and dialectal variations to boost generalization and performance. future efforts should prioritize expanding both data sources and fine-tuning techniques to enhance nlp tasks in low-resource languages like hausa.