IITD at WANLP 2022 Shared Task: Multilingual Multi-Granularity Network for Propaganda Detection (2022)

Abstract: We present our system for the two subtasks of the shared task on propaganda detection in Arabic, part of WANLP’2022. Subtask 1 is a multi-label classification problem to find the propaganda techniques used in a given tweet. Our system for this task uses XLM-R to predict probabilities for the target tweet to use each of the techniques. In addition to finding the techniques, subtask 2 further asks to identify the textual span for each instance of each technique that is present in the tweet; the task can be modelled as a sequence tagging problem. We use a multi-granularity network with mBERT encoder for subtask 2. Overall, our system ranks second for both subtasks (out of 14 and 3 participants, respectively). Our experimental results and analysis show that it does not help to use a much larger English corpus annotated with propaganda techniques, regardless of whether used in English or after translation to Arabic.

## introduction propaganda is information deliberately designed to promote a particular point of view and to influence the opinions or the actions of individuals or groups. with the rise of social media platforms, the circulation of propaganda is even more pronounced since it may be built upon a true fact, but exaggerated and biased to promote a particular viewpoint. various propaganda detection systems have been developed in recent years @xcite @xcite , but they all have been restricted to english due to the unavailability of labelled datasets (containing fine-grained annotations of textual spans) in other languages. to bridge this gap, the wanlp'2022 shared task on propaganda detection in arabic @xcite released a dataset of arabic tweets (we will call it aratweet) that uses 20 propaganda techniques, thus enabling research beyond english. 1 the code is released at github.com/sm354/mmgn there are two subtasks defined in this shared task for detecting the propaganda techniques used in a tweet: (1) identify the techniques present in the given arabic tweet, and (2) identify the span(s) of use of each technique along with the technique. subtask 1 can be viewed as a multi-label classification problem, where the tweet may contain any subset of the 20 propaganda techniques, even all or none of them. subtask 2 can be seen as a multilabel sequence tagging problem, where the system needs to predict the labels for each of the tokens. subtask 2 is more challenging than subtask 1 due to the increased level of detail it asks for. our subtask 1 system uses a multilingual pretrained language model, xlm-r @xcite to estimate a multinoulli distribution over the 20 propaganda techniques for a given arabic tweet. for subtask 2, we use the multi-granularity network (mgn) @xcite , but we replace the bert encoder with mbert @xcite . we call our resulting system mmgn. our systems, which use only aratweet data, rank second for both subtasks. we investigated cross-lingual propaganda detection by using the propaganda techniques corpus (ptc) @xcite , which consists of annotated english news articles. we trained mmgn on ptc and continued its training on aratweet. surprisingly, we found that continued training hurts the model by 10.2 f1 points absolute. to alleviate the possibility of ineffective transfer from english in mbert embeddings, we further translated the ptc to arabic using google translate and we projected the span-labels using awesome-align @xcite . upon doing continued training with a subset of the translated data, having only sentences containing propaganda, we found that it does not help, but also does not hurt the model. we believe that the domain difference between the two dataset is quite large, and thus there are no benefits in cross-lingual transfer. the dataset released in this shared task, which we call aratweet, comprises arabic tweets, most of which (but not all) contain some propaganda techniques. ## system description subtask 1 is a multi-label classification problem, where the model needs to find which of the 20 propaganda techniques (if any) are present in the input tweet. our system (shown in figure 1 ) finetunes a multilingual pretrained language model, xlm-r, @xcite for this subtask. given an arabic tweet, we first tokenize it into word pieces [t 1 , t 2 , . . . , t n ] using the xlm-r tokenizer. we the pass these pieces through xlm-r to obtain contextualised embeddings, from which we take the cls token embedding and we pass it through a single fully-connected linear layer to obtain a 20-dimensional embedding. after passing it through a sigmoid non-linearity, we convert this embedding, representing logits, to probabilities [p 1 , p 2 , ..., p 20 ], one for each propaganda technique. using a threshold of 0.5, our system assigns label i if p @xmath0.5. when p @xmath1.5 ∀ i, the model predicts no technique for the target tweet. subtask 2 is a multi-label sequence tagging problem, where we want to label the tokens of a given tweet with the propaganda techniques. since the (training) data contains tweets that do not contain propaganda (as discussed in section 2), we use the multi-granularity network (mgn) @xcite to develop our subtask 2 system. mgn uses bert @xcite and models the task as a single-label sequence tagging problem, where either one of 20 techniques or none of them is assigned to each token. to improve the performance, it also adds a trainable gate to lower the probabilities for all tokens if the sentence does not contain propaganda. 3 we replace bert with mbert in our mgn system, to obtain our multilingual multi-granularity network (mmgn) as our subtask 2 system. mmgn can work for arabic and for all other languages that are supported by mbert. ## experiments for evaluation, we use the official scorers that were released for the shared task. the official evaluation measure for subtask 1 is micro-f1. however, the scorer also reports macro-f1. for subtask 2, a modified micro-averaged f1 score is used, which gives credit to partial matches between the gold and the predicted spans. we use the dev partition of aratweet to find the best model checkpoint and to report the scores on the finally released test set. our models are trained on a single v100 (32gb) gpu. ## conclusion we described our systems for the two subtasks of the wanlp 2022 shared task on propaganda detection in arabic. for subtask 1, we used xlm-r to estimate a multinoulli distribution over the 20 propaganda techniques for multi-label classification. for subtask 2, we used a multi-granularity network with mbert, addressing the subtask as a sequence tagging problem. the official evaluation results put our systems as second on both subtasks, out of 14 and of 3 participants, respectively. we further described a number of experiments, which suggest various research challenges for future work, such as how to effectively use data from different domains, and how to learn language-agnostic embeddings for propaganda detection.