FPAI at S em E val-2021 Task 6: BERT - MRC for Propaganda Techniques Detection (2021)

Abstract: The objective of subtask 2 of SemEval-2021 Task 6 is to identify techniques used together with the span(s) of text covered by each technique. This paper describes the system and model we developed for the task. We first propose a pipeline system to identify spans, then to classify the technique in the input sequence. But it severely suffers from handling the overlapping in nested span. Then we propose to formulize the task as a question answering task by MRC framework which achieves a better result compared to the pipeline method. Moreover, data augmentation and loss design techniques are also explored to alleviate the problem of data sparse and imbalance. Finally, we attain the 3rd place in the final evaluation phase.

## introduction fake news detection has attracted the attention of many researchers. but most of the conventional fake news detection methods focuse on long-form journalism, and little attention has been paid to the propaganda techniques. memes have become the most popular type of content on social media platforms, and it can easily mislead the audience to agree with the speaker through propaganda techniques. the semeval-2021 task 6 focuses on detecting the use of rhetorical and psychological techniques in memes without (subtask 1, subtask 2) and with (subtask 3) visual content. we first adopted a model to identify the span and techniques sequentially, and made many attempts to optimize the model, but the results were not satisfactory. then we tried some end-to-end method, e.g. mrc framework. these authors contributed equally to this work this work is licensed under a creative commons attribution 4.0 international license. license details: http:// creativecommons.org/licenses/by/4.0/ we found that the pipeline model can't handle the span overlapping issue effectively, but the mrc framework with an informative query is well performing in this scenario. in addition, the data augmentation and a carefully designed loss can alleviate the impact of data sparsity and imbalance. we attain an f1 score of 0.3974 and the 3 rd place in the final evaluation phase ## related work there was also a related previous task on finegrained propaganda detection @xcite , where the participants used transformer-style models, lstms and ensembles @xcite @xcite . some approaches further used non-contextualized word embeddings, e.g., based on fasttext and glove @xcite , or handcrafted features such as liwc, quotes and questions @xcite . moreover, martino et al.2020 analysed computational propaganda detection from text perspective and network perspective, argued for the need of combined efforts blending natural language processing, network analysis, and machine learning. ## experiment and result we use the provided training dataset for training, and evaluate the model on the development dataset during evaluation phase. we implemented several experiments to explore the effect of different methods. the following is the detail for each experiment. ## conclusion we create a pipeline model and an end-to-end mrc model to identify the span and techniques. we attain an f1 score of 0.3974 and the 3 rd place at in final evaluation phase with bert-mrc @xmath0. our implementation shows that 1) reformulating the task into an mrc task to detect the overlapping span is effective; 2) data augmentation is about to increase model generalization ; 3) careful loss design can alleviate the effect of data imbalance.