Trinity at S em E val-2023 Task 12: Sentiment Analysis for Low-resource A frican Languages using T witter Dataset (2023)

Abstract: In this paper, we have performed sentiment analysis on three African languages (Hausa, Swahili, and Yoruba). We used various deep learning and traditional models paired with a vectorizer for classification and data -preprocessing. We have also used a few data oversampling methods to handle the imbalanced text data. Thus, we could analyze the performance of those models in all the languages by using weighted and macro F1 scores as evaluation metrics.

## introduction most languages are becoming digitally accessible as a result of the growing usage of the internet and social media platforms. this enables numerous artificial intelligence (ai) applications that make it possible to do operations like sentiment analysis, machine translation, and the detection of offensive information. 2,058 or 30 percent of all surviving languages, according to @xcite , are african languages. the majority of these languages, however, lack curated datasets for creating these ai applications. the lacuna fund is one recent example of a financed program that aims to buck this trend and provide such datasets for african languages. to maximise the use of these datasets, new natural language processing (nlp) techniques must be developed, and research is needed to ascertain whether the present approaches are appropriate in both cases. the shared task covers 17 african languages namely hausa, yoruba, igbo, nigerian pidgin from nigeria, amharic, tigrinya, and oromo from ethiopia, swahili from kenya and tanzania, algerian arabic dialect from algeria, kinyarwanda from rwanda, twi from ghana, mozam-bique portuguese from mozambique and moroccan arabic/darija from morocco @xcite . the major system strategies used here include using some traditional classifier models like logistic regression, svm, and randomforest. they were used through various pipelines. these pipelines also included a vectorizer and over-sampling tools like ros and smote. one more result per language was obtained using the mbert model. all the results obtained were compared and some conclusions can be drawn based on that analysis. the evaluation metrics used in this shared task were average-macro and average-weighted f1 scores. since in the shared task, afrisenti semeval 2023 we had to deal with african languages it helped us gain familiarity with handling multi-lingual data. we explored and researched different models and data-preprocessing techniques available for such kind of sentiment analysis tasks. ## background here, we have used the dataset provided in the shared task itself. thus, the input contains tweets to be classified in respective languages. we have used a vectorizer in our pipeline for data-preprocessing purposes. the output is the classification of that particular tweet into 'positive', 'negative', or 'neutral' sentiments. the tracks we participated in include languages hausa (track 1), swahili (track 8), and yoruba (track 2). ## system overview in this shared task, we have used the following models and algorithms: ## results table-1 contains the two types of f1-scores i.e. weighted and macro for the various models on which test data of the hausa language was tested. from the data we see that multilingual bert performed best among four models (some with different types of sampling) used for testing by giving a value of 76.53 and 76.55 percentage on weighted and macro types of f1-score respectively. table-2 contains the two types of f1-scores i.e. weighted and macro for the various models on which test data of the swahili language was tested. from the data we see that logistic regression (smote) performed best among four models (some with different types of sampling) used for testing by giving a value of 57.79 and 48.59 percentage on weighted and macro types of f1-score respectively. table-3 contains the two types of f1-scores i.e. weighted and macro for the various models on which test data of the yoruba language was tested. from the data we see that svc (smote) performed best among four models (some with different types of sampling) used for testing by giving a value of 74.83 and 73.15 percentage on weighted and macro types of f1-score respectively. it is hereby informed that we only submitted mbert based results in the competition and rest of the models were added later. on the basis of submitted results, we ranked 24th in hausa, 31st in yoruba and 29th in swahili tracks of subtask-1. ## conclusion thus, we implemented various traditional and transformer models for sentiment analysis of african languages. we had different types of data preprocessing techniques in case of traditional methods and analyzed their performance. it was observed that in case of a balanced dataset, deep learning based models like mbert will be more efficient. if the data is imbalanced, traditional classification and oversampling methods produce better performance. in future, we plan to train our models using on a dataset built by data scraping and data crawling. we also plan to try different tokenizers for better preprocessing.