L e C un at S em E val-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation (2021)

Abstract: We developed a system for task 6 sub-task 1 for detecting propaganda in memes. An external dataset and augmentation data-set were used to extend the official competition data-set. Data augmentation techniques were applied on the external data-set and competition data-set to come up with the augmented data-set. We trained 5 transformers (DeBERTa, and 4 RoBERTa) and ensembled them to make the prediction. We trained 1 RoBERTa model initially on the augmented data-set for a few epochs and then fine-tuned it on the competition data-set which improved the f1-micro up to 0.1 scores. After that, another initial RoBERTa model was trained on the external data-set merged with the augmented data-set for few epochs and fine-tuned it on the competition data-set. Furthermore, we ensembled the initial models with the models after fine-tuning. For the final model in the ensemble, we trained a DeBERTa model on the augmented data-set without fine-tuning it on the competition data-set. Finally, we averaged the output of each model in the ensemble to make the prediction.

## introduction the definition of memes was constantly changing since it was first conceived. but, memes eventually got an academic definition, called an "internet meme". as @xcite internet meme can roughly be defined as "a piece of culture, typically a joke, which gains influence through online transmission". but what makes internet memes unique is the speed of their transmission and the fidelity of their form. therefore the internet meme can act as a powerful medium for persuasion techniques that preach an ideology or way of thinking. @xcite on the other hand, the term "propaganda" is defined as a form of communication that employs persuasive strategies and attempts to achieve a response that furthers the desired intent of the propagandist (jowett and @xcite . with the rise of social media, a new form of propaganda rises called "computational propaganda." the author in (woolley and howard, 2017) defined computational propaganda as "the use of algorithms, automation, and human curation to purposefully distribute misleading information over social media networks". task 6 at semeval-2021 @xcite , detection of persuasion techniques in text and images, defined three subtasks. the first two subtasks deal with the textual contents of memes that ask the participants to identify which of the 20 propaganda techniques are in the text. while the third subtask encourages the participants to determine which of the 22 techniques are in the meme's textual and visual content. this paper proposes a solution for subtask1 that uses pre-trained language models to detect propaganda and possibly even identify the persuasion strategy that the propaganda sample employs. the rest of the paper is broken down as follows. section 2 discusses related-work to the task of propaganda identification. section 3 provides a description of the data and the pre-processing techniques used. section 4 describes the proposed system and architecture. section 5 presents system analysis. finally, the conclusion and future work are provided in section 6. ## related work there have been efforts in persuasion techniques identification and classification using machine and deep learning-based approaches. the authors in @xcite used word embeddings with bert @xcite and bilstm @xcite for binary detection of propaganda spans. authors in @xcite experimented with a cnn @xcite , bilstm and bert and showed bert to have the best accuracy on classifying persuasion techniques in propaganda spans. also, the authors in @xcite ) used a roberta model @xcite , a class-dependent re-weighting method and used a semi-supervised learning technique of self-training and demonstrated the effects of these techniques in an ablation study. a group of researchers @xcite experimented with a variety of plms (pre-trained language models), including bert, gpt-2 @xcite , roberta, xlm-roberta @xcite , xlnet @xcite and xlm @xcite . and have demonstrated that roberta and xlnet generally perform better for propaganda detection. ## system description different model architectures have experimented with different pre-processing techniques. the final system ended up ensembling five models, each ## conclusion and future work in this paper, we presented our proposed system, lecun, for detecting propaganda in contextual content. we have used an external dataset from a previous semeval competition and performed a data-augmentation on the external dataset to expand the dataset size. we have also investigated different ensemble combinations for state-of-theart pre-trained language models. however, there are many questions we got throughout our participation in this competition which made us curious to investigate. these questions are:-what is the influence of the data augmentation on the model performance? what is the influence of using an external dataset? how can the model weaknesses be improved? how can span identification help in improving the score of technique classification? for future work, we will be working on answering these questions. we plan to do more in-depth experimenting with different augmentation techniques and different model architectures. we will also investigate the influence of the external dataset by training models on the competition dataset, external dataset separately and compare the final results of each.