A rabic Sentiment Analysis by Pretrained Ensemble (2022)

Abstract: This paper presents the 259 team’s BERT ensemble designed for the NADI 2022 Subtask 2 (sentiment analysis) (Abdul-Mageed et al., 2022). Twitter Sentiment analysis is one of the language processing (NLP) tasks that provides a method to understand the perception and emotions of the public around specific topics. The most common research approach focuses on obtaining the tweet’s sentiment by analyzing its lexical and syntactic features. We used multiple pretrained Arabic-Bert models with a simple average ensembling and then chose the best-performing ensemble on the training dataset and ran it on the development dataset. This system ranked 3rd in Subtask 2 with a Macro-PN-F1-score of 72.49%.

## introduction sentiment analysis (sa) is a process of computationally categorizing opinions expressed in a piece of text, especially in order to determine whether the attitude towards a particular product by labeling it positive, negative, or neutral. sentiment analysis is a process of computationally categorizing opinions expressed in a piece of text, especially in order to determine whether the attitude towards a particular product by labeling it positive, negative, or neutral @xcite . as the world becomes increasingly digitized, sentiment analysis is becoming increasingly important. with the vast majority of people now using the internet and social media to communicate, nlp methods can help us analyze this massive amount of data to understand public opinion on various issues. sentiment analysis can be extremely useful for businesses and governments, using sentiment analysis to track how the public's opinion @xcite . one of the basic sentiment analysis approaches is a lexicon-based approach. this approach uses a list of words associated with neutral, positive, or negative sentiment. then we use the generated list to score the sentiment of a text similar to what is done in @xcite and @xcite . another common approach is to use a machine learning algorithm to learn the sentiment of a text. however, this approach requires a training dataset of texts manually labeled with their sentiment for the machine learning algorithm to predict the sentiment of new texts such as @xcite . recently, transformers have been used for sentiment analysis, a deep learning method that learns the representation of text data for sentiment classification. this approach has been shown to outperform traditional machine learning methods such as support vector machines or deep learning models like long short-term memory or convolutional neural networks. in addition, the transformer approach can also handle a large amount of data, making it a scalable method for sentiment analysis like @xcite . while the accuracy of sentiment analysis in arabic is still far from perfect, the current state of the art is much better than it was even a few years ago. the progress of sentiment analysis in arabic has been significant in recent years. with the increasing availability of arabic text data, there has been a corresponding increase in the development of methods and tools for sentiment analysis in arabic. this progress will likely continue as more arabicspecific data, and sophisticated methods become available @xcite . there are a few reasons why sentiment analysis is complex with arabic dialects. first, many variations in arabic dialects make it difficult to identify patterns. second, arabic is a highly inflected language; words can have multiple meanings depending on their context in a sentence. this can make it difficult to determine the sentiment. finally, arabic dialects often use a lot of idiomatic expressions, which can also be challenging to interpret, as shown in @xcite . the paper is structured as follows: section 2 concisely describes the used dataset. section 3 describes the models used for the ensemble for sentiment analysis. section 4 presents the results obtained for each combination. section 5 presents related works. section 6 presents a general discussion. finally, section 7 contains the conclusion and points for future work. ## data the competition provided a dataset of 5,000 tweets split into a training dataset of 1500 tweets, a development dataset of 500 tweets, and 3000 for the testing dataset. the tweets are labeled 'pos' for positive, 'neg' for negative, and 'neut' for neutral. we used the training dataset to decide on the best ensemble combination of models. ## system description we used arabic pretrained language models that were fine-tuned for sentiment analysis publicly available on huggingface. there was no extra fine-tuning or preprocessing done after that. instead, these models were used in a simple average ensemble by adding the logit values of the models' combination, using the maximum value for prediction, and then looping over the different combinations of the available models. ## results we have validated the different ensemble combinations models on the training dataset. the ensemble with model camelbert mix sa model and camelbert msa (modern standard arabic) sa model based on bert achieved the best results. we believe this is because the combination of different dialects in the mix model with the modern standard arabic version has the majority of text features among all other models. this ensemble probably achieved the best results because the task's data contains both msa and dialects. ## discussion it is interesting to see that the models alone can achieve good results. however, the model with just classical arabic was able to achieve no trivial result with only 5% of the size of the mix model, which is worth investigating to understand the reason behind the result. the current pretraining models available freely have a relatively good performance on this task even with such a limited dataset and no training, which shows researchers' quality and hard work in the arabic nlp field. future work would involve fine-tuning these models on the training dataset, trying different ensemble methods, and trying few-shot learning. another thing to explore would be to try arat5, the most recent state-of-the-art arabic natural language understanding system @xcite . the paper @xcite proposed a simple and effective transfer learning approach and evaluated the approach on arabic, finding that the approach outperforms the state-of-the-art on all tasks in the benchmark. tables 2 and 3 show the results obtained over development data for the nadi task (pos being positive, neg being negative, and neut being neutral) for the single model and ensemble model. five language models were used to classify sentiment (mix, ca, da, msa, and marbert). a simple two models ensemble (mix and msa) obtained the best results for the task and was selected for the final submission.