N o P ropaganda at S em E val-2020 Task 11: A Borrowed Approach to Sequence Tagging and Text Classification (2020)

Abstract: This paper describes our contribution to SemEval-2020 Task 11: Detection Of Propaganda Techniques In News Articles. We start with simple LSTM baselines and move to an autoregressive transformer decoder to predict long continuous propaganda spans for the first subtask. We also adopt an approach from relation extraction by enveloping spans mentioned above with special tokens for the second subtask of propaganda technique classification. Our models report an F-score of 44.6% and a micro-averaged F-score of 58.2% for those tasks accordingly.

## introduction in recent years natural language processing has experienced rapid development. in particular, application of deep learning to nlp @xcite , the introduction of pre-trained context-independent word embeddings such as word2vec @xcite , glove @xcite and fasttext @xcite , and utilization of rnn-based @xcite pipelines such as charcnn-blstm-crf @xcite allowed to improve sota for an overwhelming majority of nlp tasks. in 2018 a monumental breakthrough happened when context-dependent embeddings based on pre-trained language models emerged, such as elmo @xcite , ulmfit @xcite , and bert @xcite . the resulting boost in the models' performance cannot be underestimated, amounting to a 30% relative error reduction rate. researchers compare this latest development to the introduction of imagenet to computer vision. furthermore, as breakthroughs in cv allowed such technologies as deepfake to arise, similar processes are happening to nlp: the authors of gpt2 @xcite anticipated that their model could be used for malicious purposes such as fake news generation or even impersonation. given that at present, the problem of media swaying public opinion to the benefit of certain parties is more acute than ever, we must keep our news fair and unbiased; thus, techniques for fake news and propaganda detection need to be developed. semeval-2020 task 11 is an essential step in that direction. another noteworthy consequence of contextualized embeddings introduction is the generalization of models. while many models are developed for a particular task, they can be effectively utilized in order to solve other tasks. our main contribution is the utilization of two such models for new tasks, namely lasertagger @xcite , initially developed for summarization, which is used for span identification, while r-bert @xcite , originally developed for relation extraction, is used for span classification. section 2 contains the description of our systems foot_0 , section 3 contains evaluation results and error analysis, while part 4 contains the concluding remark. ## system description all of the following models based on bert @xcite use bert base pretrained model weights. hyperparameter tuning and cross-validation was performed on 5 folds split by articles. the data used for training and evaluation consists of various news articles and is described in detail in @xcite . the tasks will be briefly explained in the following sections; their full description can be found in the @xcite . ## conclusion in this work, we propose to use two models: autoregressive transformer decoder for sequence tagging and r-bert-like approach for propaganda spans technique classification. the former is taken from such tasks as sentence fusion, sentence splitting, and abstractive summarization, while the latter was originally used for relation extraction. instead of exploring various pretrained transformers like bert large , roberta, albert and xlnet or engineering domain-specific features and approaches, we focused on improving architectures and trying to push them as far as possible. in the future the results of this task may be used for propaganda detection in various media as well as unbiasing texts and making them fair.