P rta: A System to Support the Analysis of Propaganda Techniques in the News (2020)

Abstract: Recent events, such as the 2016 US Presidential Campaign, Brexit and the COVID-19 “infodemic”, have brought into the spotlight the dangers of online disinformation. There has been a lot of research focusing on fact-checking and disinformation detection. However, little attention has been paid to the specific rhetorical and psychological techniques used to convey propaganda messages. Revealing the use of such techniques can help promote media literacy and critical thinking, and eventually contribute to limiting the impact of “fake news” and disinformation campaigns. Prta (Propaganda Persuasion Techniques Analyzer) allows users to explore the articles crawled on a regular basis by highlighting the spans in which propaganda techniques occur and to compare them on the basis of their use of propaganda techniques. The system further reports statistics about the use of such techniques, overall and over time, or according to filtering criteria specified by the user based on time interval, keywords, and/or political orientation of the media. Moreover, it allows users to analyze any text or URL through a dedicated interface or via an API. The system is available online: https://www.tanbih.org/prta .

## introduction brexit and the 2016 us presidential campaign @xcite , as well as major events such the covid-19 outbreak (world health organization, 2020), were marked by disinformation campaigns at an unprecedented scale. this has brought the public attention to the problem, which became known under the name "fake news". even though declared word of the year 2017 by collins dictionary, foot_0 we find that term unhelpful, as it can easily mislead people, and even fact-checking organizations, to only focus on the veracity aspect. at the eu level, a more precise term is preferred, disinformation, which refers to information that is both (i) false, and (ii) intents to harm. the oftenignored aspect (ii) is the main reasons why disinformation has become an important issue, namely because the news was weaponized. another aspect that has been largely ignored is the mechanism through which disinformation is being conveyed: using propaganda techniques. propaganda can be defined as (i) trying to influence somebody's opinion, and (ii) doing so on purpose @xcite . note that this definition is orthogonal to that of disinformation: propagandist news can be both true and false, and it can be both harmful and harmless (it could even be good). here our focus is on the propaganda techniques: on their typology and use in the news. propaganda messages are conveyed via specific rhetorical and psychological techniques, ranging from leveraging on emotions -such as using loaded language @xcite , flag waving @xcite , appeal to authority @xcite , slogans @xcite , and clichés @xcite -to using logical fallacies -such as straw men @xcite (misrepresenting someone's opinion), red herring @xcite , @xcite (presenting irrelevant data), black-and-white fallacy @xcite (presenting two alternatives as the only possibilities), and whataboutism @xcite . here, we present prta -the propaganda persuasion techniques analyzer. prta makes online readers aware of propaganda by automatically detecting the text fragments in which propaganda techniques are being used as well as the type of propaganda technique in use. we believe that revealing the use of such techniques can help promote media literacy and critical thinking, and eventually contribute to limiting the impact of "fake news" and disinformation campaigns. technique • snippet loaded language • outrage as donald trump suggests injecting disinfectant to kill virus. name calling, labeling • who: coronavirus emergency is 'public enemy number 1' repetition • i still have a dream. it is a dream deeply rooted in the american dream. i have a dream that . . . exaggeration, minimization • coronavirus 'risk to the american people remains very low', trump said. doubt • can the same be said for the obama administration? appeal to fear/prejudice • a dark, impenetrable and "irreversible" winter of persecution of the faithful by their own shepherds will fall. flag-waving • mueller attempts to stop the will of we the people!!! it's time to jail mueller. causal oversimplification • if france had not have declared war on germany then world war ii would have never happened. slogans • "build the wall!" trump tweeted. appeal to authority • monsignor jean-franc ¸ois lantheaume, who served as first counsellor of the nunciature in washington, confirmed that "viganò said the truth. that's all." black-and-white fallacy • francis said these words: "everyone is guilty for the good he could have done and did not do . . . if we do not oppose evil, we tacitly feed it." obfuscation, intentional vagueness, confusion • women and men are physically and emotionally different. the sexes are not "equal," then, and therefore the law should not pretend that we are! thought-terminating cliches • i do not really see any problems there. marx is the president. whataboutism • president trump -who himself avoided national military service in the 1960's-keeps beating the war drums over north korea. reductio ad hitlerum • "vichy journalism," a term which now fits so much of the mainstream media. it collaborates in the same way that the vichy government in france collaborated with the nazis. red herring • "you may claim that the death penalty is an ineffective deterrent against crime -but what about the victims of crime? how do you think surviving family members feel when they see the man who murdered their son kept in prison at their expense? is it right that they should pay for their son's murderer to be fed and housed?" bandwagon • he tweeted, "eu no longer considers #hamas a terrorist group. time for us to do same." straw man • "take it seriously, but with a large grain of salt." which is just allen's more nuanced way of saying: "don't believe it." with prta, users can explore the contents of articles about a number of topics, crawled from a variety of sources and updated on a regular basis, and to compare them on the basis of their use of propaganda techniques. the application reports overall statistics about the occurrence of such techniques, as well as their usage over time, or according to user-defined filtering criteria such as time span, keywords, and/or political orientation of the media. furthermore, the application allows users to input and to analyze any text or url of interest; this is also possible via an api, which allows other applications to be built on top of the system. prta relies on a supervised multi-granularity gated bert-based model, which we train on a corpus of news articles annotated at the fragment level with 18 propaganda techniques, a total of 350k word tokens @xcite . our work is in contrast to previous efforts, where propaganda has been tackled primarily at the article level @xcite @xcite . it is also different from work in the related field of computational argumentation, which deals with some specific logical fallacies related to propaganda, such as ad hominem fallacy @xcite . consider the game argotario, which educates people to recognize and create fallacies such as ad hominem, red herring and irrelevant authority, which directly relate to propaganda @xcite . unlike them, we have a richer inventory of techniques and we show them in the context of actual news. the remainder of this paper is organized as follows. section 2 introduces the machine learning model at the core of the prta system. section 3 sketches the full architecture of prta, with focus on the process of collection and processing of the input articles. section 4 describes the system interface and its functionality, and presents some examples. section 5 draws conclusions and discusses possible directions for future work. ## data and model data we train our model on a corpus of 350k tokens @xcite , manually annotated by professional annotators with the instances of use of eighteen propaganda techniques. see slc sentence-level classification. given a sentence, predict whether it contains at least one propaganda technique. our model adds on top of bert @xcite a set of layers that combine information from the fragment-and the sentence-level annotations to boost the performance of the flc task on the basis of the slc task. the network architecture is shown in figure 1 , and we refer to it as a multigranularity network. it features 19 output units for each input token in the flc task, standing for one of the 18 propaganda techniques or "no technique." a complementary output focuses on the slc task, which is used to generate, through a trainable gate, a weight w that is multiplied by the input of the flc task. the gate consists of a projection layer to one dimension and an activation function. the effect of this modeling is that if the sentence-level classifier is confident that the sentence does not contain propaganda, i.e., w ∼ 0, then no propaganda technique would be predicted for any of the word tokens in the sentence. the model we use in prta outperforms bertbased baselines on both at the sentence-level (f 1 of 60.71 vs. 57.74) and at the fragment-level (f 1 of 22.58 vs. 21.39). at the fragment-level, the model outperforms the best solution of a hackathon organized on this data. 3 for the prta system, we applied a softmax operator to turn its output into a bounded value in the range [0,1], which allows us to show a confidence for each prediction. further details about the techniques, the model, the data, and the experiments can be found in @xcite . 4 3 system architecture prta collects news articles from a number of news outlets, discards near-duplicates and finally identifies both specific propaganda techniques and sentences containing propaganda. we crawl a growing list (now 250) of rss feeds, twitter accounts, and websites, and we extract the plain text from the crawled web pages using the newspaper3k library foot_4 . we then perform deduplication based on a combination of url partial matching and content analysis using a hash function. finally, we use the model from section 2 to identify sentences with propaganda and instances of use of specific propaganda techniques in the text and their types. we further organize the articles into topics; currently, the topics are defined using keyword matching, e.g., an article mentioning covid-19 or brexit is assigned to a corresponding topic. by accumulating the techniques identified in multiple articles, prta can show the volume of propaganda techniques used by each medium -as well as aggregated over all media for a specific topic-thus, allowing the user to do comparisons and analysis, as described in the next section. ## interface prta offers the following functionality. for each crawled news article: ## conclusion and future work we have presented the prta system for detecting and highlighting the use of propaganda techniques in online news. the system further shows aggregated statistics about the use of such techniques in articles filtered according to several criteria, including date ranges, media sources, bias of the sources, and keyword searches. the system also allows users to analyze their own text or the contents of a url of interest. we have made publicly available our data and models, as well as an api to the live system. we hope that the prta system would help raise awareness about the use of propaganda techniques in the news, thus promoting media literacy and critical thinking, which are arguably the best longterm answer to "fake news" and disinformation. in future work, we plan to add more media sources, especially from non-english media and regions. we further want to extend the tool to support other propaganda techniques such as cherrypicking and omission, among others, which would require analysis beyond the text of a single article.