UCAS - IIE - NLP at S em E val-2023 Task 12: Enhancing Generalization of Multilingual BERT for Low-resource Sentiment Analysis (2023)

Abstract: This paper describes our system designed for SemEval-2023 Task 12: Sentiment analysis for African languages. The challenge faced by this task is the scarcity of labeled data and linguistic resources in low-resource settings. To alleviate these, we propose a generalized multilingual system SACL-XLMR for sentiment analysis on low-resource languages. Specifically, we design a lexicon-based multilingual BERT to facilitate language adaptation and sentiment-aware representation learning. Besides, we apply a supervised adversarial contrastive learning technique to learn sentiment-spread structured representations and enhance model generalization. Our system achieved competitive results, largely outperforming baselines on both multilingual and zero-shot sentiment classification subtasks. Notably, the system obtained the 1st rank on the zero-shot classification subtask in the official ranking. Extensive experiments demonstrate the effectiveness of our system.

## introduction sentiment analysis is a critical aspect of natural language processing with numerous applications, including public opinion monitoring @xcite , healthcare services @xcite , and recommendation systems @xcite . however, performing sentiment analysis in low-resource languages poses significant challenges, including the scarcity of labeled data and linguistic resources, as well as the diversity of languages and dialects @xcite . in semeval-2023 task 12 @xcite , the focus is on sentiment analysis for african languages in twitter, which further exacerbates the challenges due to the presence of tone, code-switching, and digraphia phenomena @xcite . although multilingual pre-trained language models (multilingual ptms) @xcite have shown potential in cross-lingual transfer learning compared to monolingual ptms @xcite , they have limitations in capturing nuances and cultural differences within a language, especially in the context of dialects and regional variations. in this paper, we propose a generalized multilingual system named sacl-xlmr to address these limitations and enhance the generalization of multilingual ptms for under-represented languages, particularly african languages. our system leverages a lexicon-based multilingual bert model to facilitate language adaptation and sentimentaware representation learning. additionally, we apply a supervised adversarial contrastive learning (sacl) technique @xcite to learn sentiment-spread structured representations and enhance model generalization. we present the details of the proposed system and evaluate its performance on semeval-2023 task 12. our system achieves remarkable performance, outperforming baselines by +1.1% weighted-f1 score on multilingual sentiment classification subtask and by +2.8% weighted-f1 score on zero-shot sentiment classification subtask in the afrisenti-semeval datasets @xcite . moreover, following the afrisenti se-meval prizes foot_0 and the task description (muhammad et al., 2023b), our system obtains the 1st rank on the zero-shot classification subtask in the official ranking. we conducted experiments to demonstrate the effectiveness of our approach, highlighting the potential of our system in overcoming the challenges of low-resource sentiment analysis. the afrisenti datasets 2 (muhammad et al., 2023a) are a collection of multilingual twitter datasets that consist of 110,000+ tweets in 14 lowresource african languages from four language 2 https://github.com/afrisenti-semeval families for sentiment analysis. the statistics of each monolingual tweet datasets are reported in ## system overview in this section, we describe our system adopted in semeval-2023 task 12, where we design a generalized multilingual system named sacl-xlmr for sentiment analysis on low-resource languages. the overall architecture is illustrated in figure 1 . with the guidance of sacl framework, the method can learn label-consistent structured features for better sentiment classification. ## conclusion in this paper, a multilingual system named sacl-xlmr has been proposed for sentiment analysis on low-resource african languages. the system employs a lexicon-based multilingual bert to facilitate language adaptation and sentiment-aware representation learning. it also uses a supervised adversarial contrastive learning technique to learn sentiment-spread structured representations and enhance model generalization. the system achieved competitive results, largely outperforming the comparison baselines on both multilingual and zeroshot sentiment classification subtasks, and obtained the 1st rank on zero-shot classification subtask in the official ranking.