G unadarma XBRIN at S em E val-2023 Task 12: Utilization of SVM and A fri BERT a for Monolingual, Multilingual, and Zero-shot Sentiment Analysis in A frican Languages (2023)

Abstract: This paper describes our participation in Task 12: AfriSenti-SemEval 2023, i.e., track 12 of subtask A, track 16 of subtask B, and track 18 of subtask C. To deal with these three tracks, we utilize Support Vector Machine (SVM) + One vs Rest, SVM + One vs Rest with SMOTE, and AfriBERTa-large models. In particular, our SVM + One vs Rest with SMOTE model could obtain the highest weighted F1-Score for tracks 16 and 18 in the evaluation phase, that is, 65.14% and 33.49%, respectively. Meanwhile, our SVM + One vs Rest model could perform better than other models for track 12 in the evaluation phase.

## introduction sentiment analysis (sa) is one of the natural language processing (nlp) tasks. sa is used to recognize and classify the author's opinion or experience of an object, such as a specific topic, product, service, organization, social events, politics, and economic @xcite @xcite . the sentiment of the opinion could be classified as positive, neutral, or negative. implementation of sa is employed in various sectors, such as business, healthcare, and education @xcite . however, sa research mostly focused on high-resource language, i.e., english and chinese @xcite . in other words, low-resource language is still not yet addressed well @xcite , such as african language @xcite , which is the focus of task 12. task 12 afrisenti-semeval 1 @xcite aims to do sentiment analysis on african 1 https://codalab.lisn.upsaclay.fr/ competitions/7320 datasets collected from twitter. this task 12 consists of three subtasks, i.e., monolingual sentiment classification (subtask a), multilingual sentiment classification (subtask b), and zero-shot sentiment classification (subtask c). in total, there are 14 datasets provided in the task 12, namely, hausa, yoruba, igbo, nigerian pidgin, amharic, algerian arabic, moroccan arabic/darija, swahili, kinyarwanda, twi, mozambican portuguese, xitsonga (mozambique dialect), tigrinya, and oromo. in this afrisenti-semeval task, we participate in track 12: xitsonga (mozambique dialect) of subtask a, track 16: 12 languages in subtask a of subtask b, and track 18: zero-shot on oromo of subtask c. to deal with each subtask, we utilize three models, namely, support vector machine (svm) + one vs rest (model 1), svm + one vs rest with smote (model 2), and afribertalarge @xcite (model 3). the highestobtained f1-scores of our participation are 50.56%, 65.14%, and 33.49% for track 12, track 16, and track 18, respectively. the code used in this work is available on github foot_0 . ## method details of preprocessing, models, and evaluation employed in this work are as follows: ## results based on the results in overall, our svm + one vs rest (with or without smote) could perform better than our afriberta-large in the evaluation phase of tracks 12, 16, and 18. sadly, our best-obtained results are still unsatisfactory, even far below the results obtained by the highest-ranking team in tracks 12, 16, and 18. this shortcoming might be due to the dataset characteristic. for example, we found that several hausa tweets with positive sentiments in the evaluation dataset of track 16 were misclassified by our models as negative ones. to analyze this misclassification, we extracted the positive and negative words in the tweets using the lexicon sentiment provided by afrisenti foot_11 . the extraction result shows that several positive tweets are containing negative words, but positive words do not exist (see ## conclusion we utilized svm + one vs rest, svm + one vs rest with smote, and afriberta-large to deal with tracks 12, 16, and 18 of the afrisenti-semeval task. our highest-obtained weighted f1-scores were 50.56%, 65.14%, and 33.49% for tracks 12, 16, and 18, respectively. although our obtained results indicated that our models could not perform as well as or better than the baseline models, the svm + one vs rest (with or without smote) results could be compared with those of the baseline ones. for further work, it will be better to explore hyperparameter tuning on the pre-trained model to improve the model performance. in addition, there are several tweets written in latin script. thus, it may be beneficial to translate arabic into latin and vice versa when dealing with this task in the future.