SAIDS : A Novel Approach for Sentiment Analysis Informed of Dialect and Sarcasm (2022)

Abstract: Sentiment analysis becomes an essential part of every social network, as it enables decision-makers to know more about users’ opinions in almost all life aspects. Despite its importance, there are multiple issues it encounters like the sentiment of the sarcastic text which is one of the main challenges of sentiment analysis. This paper tackles this challenge by introducing a novel system (SAIDS) that predicts the sentiment, sarcasm and dialect of Arabic tweets. SAIDS uses its prediction of sarcasm and dialect as known information to predict the sentiment. It uses MARBERT as a language model to generate sentence embedding, then passes it to the sarcasm and dialect models, and then the outputs of the three models are concatenated and passed to the sentiment analysis model. Multiple system design setups were experimented with and reported. SAIDS was applied to the ArSarcasm-v2 dataset where it outperforms the state-of-the-art model for the sentiment analysis task. By training all tasks together, SAIDS achieves results of 75.98 FPN, 59.09 F1-score and 71.13 F1-score for sentiment analysis, sarcasm detection, and dialect identification respectively. The system design can be used to enhance the performance of any task which is dependent on other tasks.

## introduction sentiment analysis (sa) is one of the main tasks in the natural language processing (nlp) field. it is used for opinion mining which supports decisionmakers. working on sentiment analysis starts relatively early, for example, @xcite analysed the sentiment to positive and negative in movie reviews. following this paper, sentiment analysis becomes one of the most important topics in nlp, especially with the increasing number of reviews on websites and social media platforms. since then, a lot of work has been done in english sentiment analysis, while arabic has relatively much less. since @xcite started their work on arabic sa, multiple researchers also began theirs. now there are well-known arabic sa models like @xcite @xcite . of course, working with arabic has many challenges, one of the most challenging issues is the complex morphology of the arabic language @xcite . another challenge is the variety of arabic dialects @xcite . moreover, one of the wellknown challenges in sa for all languages is sarcasm, as the sarcastic person uses words and means the opposite of it. for example, "i'd really truly love going out in this weather!", does it reflect a positive or negative sentiment? because of the sarcasm, we cannot judge the sentiment correctly. several related works tackle english sarcasm detection with sentiment analysis @xcite @xcite . on the other hand, there are only a few works on both sentiment and sarcasm in arabic. there are two shared tasks on sarcasm detection @xcite , but for both sarcasm and sentiment there was only one shared task @xcite but each sub-task is independent, meaning that participating teams can submit a different model for each task. some participants used the same model for both sentiment and sarcasm @xcite . instead of training sentiment independently of sarcasm, this work introduces a new model architecture that works with multi-task training which trains both at the same time. there are other additions to the proposed architecture; firstly, it trains with dialect also. secondly, the sarcasm and dialect that are initially predicted are used in the prediction of the sentiment. in other words, the sentiment model is informed by the sarcasm and dialect model output. the contributions offered by this work are: this paper is organized as follows section 2 shows the related work on sentiment analysis, sarcasm detection, and dialect identification. section 3 describes the dataset used in this work and shows data statistics. section 4 describes saids model and all the design setups. section 5 shows the experimental results and finally section 6 concludes the work. ## dataset arsarcasm-v2 @xcite is the main dataset used in this work, it was released on wanlp 2021 shared task for two tasks sarcasm and sentiment analysis. it has about 15k tweets and is divided into 12k for training and 3k for testing, the same test set, as released on wanlp 2021, was used. each tweet was labelled for the sentiment (positive (pos), neutral (neu), and negative (neg)), sarcasm (true, and false), and dialect (msa, egypt (egy), levantine (lev), maghreb (nor), and gulf). the authors of the dataset annotate it using a crowd-sourcing platform. this dataset originally consisted of a combination of two datasets, the first one is arsarcasm @xcite ) and the second one is daict @xcite merged the two datasets. ## proposed system this section presents a detailed description of the proposed system. saids learns sentiment analysis, sarcasm detection, and dialect identification at the same time (multi-task training), in addition, it uses the sarcasm detection and dialect outputs as an additional input to the sentiment analysis model which is called "informed decision". saids decides the sentiment class using the information of sarcasm and dialect class which are both outputs itself. the main idea behind saids is based on analyzing the dataset statistics, as shown in section 3, which says that most sarcastic tweets are classified as negative tweets and most msa tweets are classified as neutral tweets. ## results in this section, the results achieved with saids are discussed. for the sake of comparison, baselines were built for the system. to initially evaluate the idea itself, a random forest model baseline was built and compared with the random forest version of saids. baselines for real scenario are baseline one (b1) which is identical to bertmodelforse-quenceclassification class in huggingface's @xcite , which takes the marbertv2 sentence embedding and passes it to the output layer for classification, and baseline two (b2) which uses two hidden layers before the classification layer, the hidden layer size is equal to the "sentiment model" hidden layer size, and baseline three (b3) which uses a larger hidden layer size to match the total number of trained parameters of saids model. for evaluation, we used the original metrics described for the dataset @xcite . for sentiment analysis, the metric is the average of the f1-score for the negative and positive classes (fpn). for sarcasm detection, the metric is f1score for the sarcastic class only (fsar). for dialect identification, we used the weighted average of the f1-score for all dialects (wfs). ## conclusion sentiment analysis is an important system that is being used extensively in decision-making, though it has different drawbacks like dealing with sarcastic sentences. in this work, we propose saids which is a novel model architecture to tackle this problem. saids essentially improves the sentiment analysis results while being informed of sarcasm and dialect of the sentence. this was achieved by training on the arsarcasm-v2 dataset which is labeled for sentiment, sarcasm, and dialect. saids's main target is to predict the sentiment of a tweet. it is trained to predict dialect and sarcasm, and then make use of them to predict the sentiment of the tweets. this means that while the model is predicting the sentiment, it is informed of its sarcasm and dialect prediction. saids achieved state-ofthe-art performance on the arsarcasm-v2 dataset for predicting the sentiment; 75.98 percent average f1-score for negative and positive sentiment. for sarcasm detection, saids achieved a 59.09 percent f1-score for the sarcastic class, whereas for dialect identification it achieved a 71.13 percent weighted f1-score for all the dialects. we believe that this model architecture could be used as a starting point to tackle every challenge in sentiment analysis. not only sentiment analysis but also this is a general architecture that can be used in any context where the prediction of a task depends on other tasks. the idea behind the architecture is intuitive, train for both tasks and inform the model of the dependent task with the output of the independent task.