DUTH at S em E val-2020 Task 11: BERT with Entity Mapping for Propaganda Classification (2020)

Abstract: This report describes the methods employed by the Democritus University of Thrace (DUTH) team for participating in SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. Our team dealt with Subtask 2: Technique Classification. We used shallow Natural Language Processing (NLP) preprocessing techniques to reduce the noise in the dataset, feature selection methods, and common supervised machine learning algorithms. Our final model is based on using the BERT system with entity mapping. To improve our model’s accuracy, we mapped certain words into five distinct categories by employing word-classes and entity recognition

## introduction according to the institute for propaganda analysis foot_0 , propaganda is an expression of an opinion or an action by individuals or groups deliberately designed to influence the opinions or the actions of other individuals or groups concerning predetermined ends. with the rapid change that the world wide web has made, it is evident that the means available for propaganda to be spread are more than ever before. the fact that, nowadays, news outlets can reach out to millions of people through their websites or social media demonstrates how easy it is to manipulate people with propaganda techniques or fake news. for example, political forecasts severely underperformed in predicting the results of the 2016 us presidential election and the united kingdom european union membership referendum (brexit) as opposed to the consensus in social media, which is indicative of the new challenges that are upon us @xcite . the semeval-2020 task 11: detection of propaganda techniques in news articles aims to produce models that can identify text fragments with various propaganda techniques. the first subtask is a binary sequence tagging task in which a model has to return the spans that contain at least one propaganda technique. the second subtask is a multi-class classification task in which given a text fragment and the article it occurs in, participants must classify the fragment into one of 14 different propaganda classes. more details on the task can be found on the task description paper @xcite . the rest of this paper is structured as follows. section 2 outlines some previous studies of propaganda identification. section 3 describes our approach, while sections 4 and 5 present experiments and results respectively. conclusions are summarized in section 6. rashkin et al. (2017) described the need for examining lexical features when trying to understand the differences between more and less reliable digital news sources. they studied the usefulness of linguistic morphology in different types of fake news such as propaganda, satire, and hoaxes. they also created a corpus of categorized news articles with labels such as propaganda, trusted, hoax, or satire. in another major study, @xcite noted the importance of discovering relationships between different propaganda techniques. they hypothesized that finding common traits could prove helpful in classification tasks. in our approach, we investigated the effects of entity mapping in certain classes, and our conclusions are in line with @xcite concerning the existence of conceptual and linguistic relationships between propaganda techniques. in recent years, there have been some significant landmarks in the nlp field. elmo @xcite , ulmfit @xcite , openai gpt @xcite , and bert @xcite are some large scale models that have massively improved the results in many nlp tasks. these systems provide models that have been pre-trained in massive corpora of unlabeled data and require fine-tuning in task-specific data. although these systems offer excellent results, there is a need for further experimentation, as noted by @xcite which highlights bert's shortcomings in real-world scenarios. ## approach this section describes our approach to mapping certain words into five distinct categories by employing word-classes and entity recognition. it also introduces the bert model which was employed for our final submission. ## experimental setup in this section, we describe the experimental setup of this study, providing information for the dataset and the parameters of machine learning algorithms, respectively. ## results this section summarizes our experimental results. before our officially submitted run, we present some additional experiments. ## conclusions we presented a supervised learning model for classifying text fragments from news articles in thirteen propaganda categories. we used standard classification techniques as well as modern nlp models such as bert. we examined the task from a sociological point of view and we tried to experiment with the fact that different entities of the same type can have the same value for propaganda classification. the results were promising and further experiments could improve them.