Discourse Structures Guided Fine-grained Propaganda Identification (2023)

Abstract: Propaganda is a form of deceptive narratives that instigate or mislead the public, usually with a political purpose. In this paper, we aim to identify propaganda in political news at two fine-grained levels: sentence-level and token-level. We observe that propaganda content is more likely to be embedded in sentences that attribute causality or assert contrast to nearby sentences, as well as seen in opinionated evaluation, speculation and discussions of future expectation. Hence, we propose to incorporate both local and global discourse structures for propaganda discovery and construct two teacher models for identifying PDTB-style discourse relations between nearby sentences and common discourse roles of sentences in a news article respectively. We further devise two methods to incorporate the two types of discourse structures for propaganda identification by either using teacher predicted probabilities as additional features or soliciting guidance in a knowledge distillation framework. Experiments on the benchmark dataset demonstrate that leveraging guidance from discourse structures can significantly improve both precision and recall of propaganda content identification.

## introduction propaganda refers to a type of misleading and deceptive information used to promote or publicize a certain political point of view @xcite @xcite . this information is often manipulated in a strategic way to shape societal beliefs @xcite . propaganda can be harmful to both individuals and society as a whole, such as disseminating false information, inciting people's perceptions, leading to conflicts, perpetuating prejudices, impeding democracy process etc. @xcite @xcite . despite its misleading nature and harmful effects, propaganda can be pervasive in political news, and has the potential to reach very large audiences @xcite . hence, developing intelligent models to identify propaganda in political news is important and necessary. instead of detecting propaganda at the level of articles @xcite @xcite , this paper focuses on identifying propaganda at fine-grained levels: sentence-level and token-level. through extracting the sentence or fragment imbued with propaganda, our aim is to accurately locate the propagandistic content and thereby provide detailed interpretable explanations. propaganda content not only presents unverified or even false information, but also employs a variety of argumentation strategies to convince the readers @xcite . therefore, identifying propaganda at fine-grained levels still remains a difficult and challenging task, and requires profound understanding of broader context in an article @xcite . first, we observe that propaganda can be injected into sentences that attribute causality or assert contrast to nearby sentences. take the article in figure 1 as an example, the second sentence (s2) makes an illogical and misleading deduction from its preceding sentence: this suggested giving advice on how to prevent jihadist attacks is now against community standards. propaganda content such as s2 usually disseminate the misinformation by leveraging causal relations, either by inferring baseless reasons or deducting speculative consequences. in addition, propaganda content can also utilize contrast relation to raise doubt or challenge credibility. for example, the last sentence (s4) casts doubts towards its previous sentence: why would facebook remove such a page and never explain why? through the strategy of contrasting, the author aims to undermine the credibility of facebook and thereby incite public protest. accordingly, we propose that understanding the discourse relations of a sentence with its nearby sentences in the local context can enable discovery of propaganda contents. moreover, we observe that propaganda content or deceptive narratives is more likely to be embedded into opinionated evaluations or expectation speculations. in contrast, sentences describing factual occurrences are less likely to carry propaganda. take the article in figure 2 as an example, the first three sentences, either reporting the main event or describing the current context triggered by the main event, all provide the readers with factual occurrences devoid of deceptive content. however, in the succeeding sentence (s4), the author includes a quotation to express emotional assessments: this reckless decision was based on her political agenda. propaganda sentences such as s4 always convince the readers and influence their mindset by inserting opinionated evaluations. furthermore, the author speculates future expectations in the next sentence (s5) that ice officers will continue to protect public safety. propaganda sentences such as s5 usually promise a bright yet unprovable future with the aim of gaining trust and support. therefore, we propose that understanding the discourse role of a sentence in telling a news story can help reveal propaganda. motivated by the above observations, we propose to incorporate both local and global discourse structures for propaganda identification. specifically, we establish two teacher models to recognize pdtb-style discourse relations between a sentence and its nearby sentences @xcite , as well as identify one of eight common news discourse roles for each sentence based upon news discourse structure @xcite . we further devise two approaches to effectively incorporate the two types of discourse structures for propaganda identification. the first approach concatenates the predicted probabilities from two teacher models as additional features. the second approach develops a more sophisticated knowledge distillation framework, where we design a response-based distillation loss to mimic the prediction behavior of teacher models, as well as a feature relation-based distillation loss to seek guidance from the embeddings generated by teacher models. the responsebased and feature relation-based distillation mutually complement each other, acquiring an enhanced guidance from discourse structures. experiments on the benchmark dataset demonstrate the effectiveness of our approaches for leveraging discourse structures, with both precision and recall improved. the ablation study validates the necessity and synergy between local and global discourse structures. ## discourse structures in this section, we explain the details for the two discourse structures: discourse relation based on pdtb-style relations, and discourse role that draws upon news discourse structure. we also perform a statistical analysis to verify our empirical observations, and introduce the procedure of constructing teacher models for both discourse structures. ## fine-grained propaganda identification in order to incorporate the two types of discourse structures into propaganda identification, we further devise two methods: a feature concatenation model and a knowledge distillation model. figure3 illustrates the framework of knowledge distillation. considering the news articles are typically long, we utilized longformer @xcite as the basic language model to encode the entire article. given a candidate propaganda article consisting of n sentences, sentence embeddings (s 1 , s 2 , ..., s n ) are initialized as the hidden state at sentence start @xmath0. the i-th sentence contains m tokens, and its tokens embeddings are (w i1 , w i2 , ...w im ). ## related work propaganda attracted research interests for years. prior work focus on detecting propaganda at articlelevel @xcite . the first work on fine-grained propaganda analysis was introduced by @xcite . a shared challenge focusing on tokenlevel tasks was launched by @xcite . several approaches have been developed for propaganda analysis, such as @xcite designed an unified neural network, @xcite utilized pretrained ensemble learning, @xcite trained a multimodal model mixing textual and visual features, and (vijayaraghavan and vosoughi, 2022) employed multi-view representations. in this paper, we focus on identifying propaganda in news articles at both sentence-level and token-level, leveraging discourse structures. misinformation detection was also studied for years, such as fake news @xcite @xcite , political bias @xcite , and logical fallacy @xcite . although propaganda may intersect with fake news, political bias, and logical fallacies, however, they are all distinct phenomena and tasks. fake news and rumor always hallucinate untruthful information. political bias refers to selectively reporting verified facts while leaving readers to arrive at their own conclusions. logical fallacies focus on errors in reasoning and argumentations to reach an invalid conclusion. in contrast, propaganda presents unverified speculation or projections in the same tone as facts, and employs a variety of persuasion strategies to convince the readers, with the purpose to manipulate public beliefs to a predetermined conclusion. media bias. in the most broad sense, propaganda news articles is a type of biased news reports. however, media bias often refers to ideological bias these days @xcite @xcite , and ideological bias is often expressed in a subtle way or under a neutral tone (van den berg and markert, 2020; @xcite by selectively including certain facts to subtly shift public opinions @xcite . in contrast, propaganda is not limited to hyper-partisan cases and can be applied to influence public beliefs in a way that aligns with the interests of the propagandist @xcite . propaganda often contains intensely emotional or opinionated content to incite or persuade the public @xcite , or presents unverified speculations, projections and deceptions @xcite . indeed, in the current media landscape, ideologically biased media sources and propaganda media sources are often labeled separately, for example, media bias/fact check foot_3 distinguishes ideologically biased sources, conspiracy theory sources, questionable sources which includes major propaganda sources, and a couple other categories. ideology bias and propaganda are studied separately as well in the nlp community @xcite , and each task features their own benchmark datasets @xcite @xcite with documents retrieved from different media sources. ## conclusion this paper aims to identify propaganda at sentencelevel and token-level. we propose to incorporate two types of discourse structures into propaganda identification: local discourse relation and global discourse role. we further design a feature concatenation model and a knowledge distillation model to leverage the guidance from discourse structures.