i C ompass at WANLP 2022 Shared Task: ARBERT and MARBERT for Multilabel Propaganda Classification of A rabic Tweets (2022)

Abstract: Arabic propaganda detection in Arabic was carried out using transformers pre-trained models ARBERT, MARBERT. They were fine-tuned for the down-stream task in hand ‘subtask 1’, multilabel classification of Arabic tweets. Submitted model was MARBERT the got 0.597 micro F1 score and got the fifth rank.

## introduction propaganda is one type of information that is shared deliberately for gaining political and religious influence. it is the systematic and deliberate way of shaping opinion and influencing the thoughts of a person for achieving the desired intention of a propagandist. in the age of "post-truth" @xcite , anti-scientific thinking and conspiracy theories the promotion of doctrines and ideologies that aim to manipulate and influence readers have rapidly spread through new communication mediums. in india, tv played a major role in the 2014 election, and some research has concluded that their results may have been swayed by propaganda techniques @xcite . furthermore, social media platforms have known a widespread of propaganda, misinformation, and hate speech in their content. during the november 2012 gaza conflict, israel defense force and hamas' alqassam brigades posted graphic images of death and suffering as well as explicit propaganda illustrations through their twitter accounts @xcite . social media platforms through their selective recommendation algorithms and their massive reach have fostered propaganda networks and "echo chambers" that amplify certain agendas and hide counter opinions and rebuttals. propaganda actions may be now more effective than ever, representing a major global risk, possibly able to influence public opinion enough to alter election outcomes, decide wars, refuse covid19 vaccines, and promote terrorism. for these reasons the need for modern automated and objective tools for uncovering propaganda is rising considerably. ## related works in the last few years research on detecting propaganda has seen a significant increase. the shared tasks found in workshops such as nlp4if 2019 @xcite and se-meval @xcite (semantic evaluation) helped accelerate research on detecting propaganda and extracting the present propaganda techniques in a sentence or in a fragment of text. also, apart from these workshops there exists work on binary classification of propaganda in the context of sentencelevel and article-level @xcite . on the other hand, arabic propaganda detection research @xcite is still lacking compared to its english counterpart @xcite . our study presented in this paper attempts to classify propaganda techniques (multilabel classification) found in textual tweets using deep learning techniques and transformer architectures such as arbert and marbert. ## system to achieve the best results we have used language models (lms) such as marbert and arbert @xcite . these models as their name suggest are based on the bert (bidirectional encoder representations from transformers) @xcite language model which is a trained transformer encoder stack that uses bidirectional self-attention and was introduced by google in 2018. while bert focuses on the english language arbert and marbert were introduced to improve arabic nlp tasks: arbert is pretrained on standard arabic language from sources such as wikipedia and books. on the other hand, marbert focuses on dialectical arabic. it is pretrained on a large database of arabic tweets on top of the bert-based models, we have used global average pooling 1d and global max pooling 1d layers. both of the pooling layers were concatenated and passed to a dropout layer and a final output layer. we have tested both arbert and marbert with and without cross-validation. cross-validation was done for 5 folds each with a percentage of 10% for the test. we also tested the models with and without the pooling layers. the training was done for 10 epochs using early stopping and we saved the best model on each epoch (according to the validation loss). ## results we evaluated each model and each configuration at least 5 times and we calculated their mean and standard deviation. the results are plotted in (fig 2 ) ## conclusion in this paper, we analyzed the performance of the pre-trained models arbert and marbert despite the small-sized annotated data and huge unbalance presented in the provided data. to obwith pooling without pooling mean std mean std arbert with cross validation 0.544 0.021 0.567 0.028 arbert without cross validation 0.548 0.03 0.559 0.025 marbert with cross validation 0.562 0.012 0.53 0.025 marbert without cross validation 0.524 0.028 0.528 0.047