H oward U niversity Computer Science at S em E val-2023 Task 12: A 2-Step System Design for Multilingual Sentiment Classification with Language Identification (2023)

Abstract: The recent release of the AfriSenti-SemEval shared Task 12 has made available 14 new datasets annotated for sentiment analysis on African Languages. We proposed and evaluated two approaches to this task, Delta TF-IDF, and a proposed Language-Specific Model Fusion Algorithm using Language Identification, both of which produced comparable or better classification performance than the current state-of-art models on this task: AfriBERTa, AfroXLMR, and AfroLM.

## introduction afrisenti-semeval shared task 12 @xcite provides an landmark dataset african language sentiment analysis. the task provided 13 datasets comprising 12 different languages, each being a dataset and a dataset composed of all the languages. the 12 african langauges covered are hausa(ha), yoruba(yo), igbo(ig), nigerian pigdin(pcm), amharic(am), algerian arabic(dz), moroccan arabic/darija(ma), swahili(sw), kinyarwanda(kr), twi(twi), mozambican portuguese(pt), and xitsonga(mozambique dialect) (ts). these dataset languages cover a wide range of different regions of africa. each tweet in each dataset is manually annotated between 3 positive, negative, and neutral labels. some datasets contain code-mixed data (data that contains two or more languages) and transliteration (converting text from one script to another that involves swapping letters). while most of these languages have a limited amount of corpus, to our knowledge, some languages, such as xitsonga, have labeled sentiment analysis datasets created for the first time @xcite . this paper aims to contribute to the nlp and sentiment analysis literature for african languages by going in-depth into our modeling practices on the afrisenti-semeval @xcite shared task 12 and showcasing our results. we propose a two-step modeling approach by first performing language detection to identify the text's language and following up with a language-specific model for final sentiment classification. we were able to discover the effectiveness of delta tf-idf on low-resource african languages. this proves useful since it requires a lot less data to be just competitive or even outperform the much larger comparable transformers models. furthermore, building on this, we were able to propose and develop a high-performing algorithm by incorporating language detection; which allowed us to leverage the best-performing models of specific languages since they provide the best performance. ## relevant works this section will cover the models that we trained and utilized to build up our proposed system design. ## design philosophy our design was driven significantly by fundamental principles of system design as well as lived experience as nlp researcher. we design our system on the basis of the following: ## system overview this section will go through our extensive modeling process. ## conclusion the recent release of the afrisenti-semeval shared task 12 has made available 14 new datasets annotated for sentiment analysis on african languages. we proposed and evaluated two approaches to this task, delta tf-idf, and a proposed language-specific model fusion algorithm using language identification, both of which produced comparable or better classification performance than the current state-of-art models on this task: afriberta, afroxlmr, and afrolm. our work aims to advance the field of sentiment analysis on african languages, which is still in its early stages. however, to get the field of research to where it needs to be, researchers must allocate the resources, care, and attention needed. with the growth in the field of sentiment analysis, it is crucial to remember it carries the risk of abuse, including large-scale surveillance and restriction of freedom of expression. therefore, responsible use of these technologies is necessary to prevent such abuses. our work's limitations include the need for more data to enhance performance, as well as the use of pre-existing models that may carry performance issues and bias. we suggest the establishment of partnerships with native speakers and increasing data access and availability from african academic institutions to overcome this limitation. additionally, using language-specific models grows linearly with the number of languages, and multilingual models may be a future solution. lastly, the authors are not experts or speakers of the languages studied, making further qualitative analysis challenging.