Towards a More In-Depth Detection of Political Framing (2023)

Abstract: In social sciences, recent years have witnessed a growing interest in applying NLP approaches to automatically detect framing in political discourse. However, most NLP studies by now focus heavily on framing effect arising from topic coverage, whereas framing effect arising from subtle usage of linguistic devices remains understudied. In a collaboration with political science researchers, we intend to investigate framing strategies in German newspaper articles on the “European Refugee Crisis”. With the goal of a more in-depth framing analysis, we not only incorporate lexical cues for shallow topic-related framing, but also propose and operationalize a variety of framing-relevant semantic and pragmatic devices, which are theoretically derived from linguistics and political science research. We demonstrate the influential role of these linguistic devices with a large-scale quantitative analysis, bringing novel insights into the linguistic properties of framing.

## introduction framing is a ubiquitous strategy to promote certain views, values or ideologies in political discourse: the information sender selectively makes certain aspects of an issue more salient in the discourse while excluding or denying the others, with the aim of ultimately influencing the public's opinions and behaviors @xcite . in recent years, automated framing detection has received increasing attention in both nlp and social sciences. in an interdisciplinary project, we are interested in identifying framing strategies employed by different german newspapers in the discourse of the event "european refugee crisis" between 2014-2018, where a large amount of asylum seekers from war-torn countries in the middle east and north africa flooded into europe. by now, most of the nlp studies on automated framing detection have been focusing on topical framing, e.g, whether the topic of economic impact or cultural value is more dominant in the discourse of migration (see, inter alia, @xcite @xcite . however, little is known about the linguistic properties of framing: in the existing nlp work, there is very few in-depth investigations on the effects of individual linguistic components in framing, which is mainly because many studies use neural networks (nns) that lack explainability and do not allow a drilling down into the effects of linguistically meaningful components (see section 2 for a detailed review). moreover, the majority of the earlier studies apply supervised approaches which rely on intense manual annotation effort. this has led to a bias towards english in the research of framing detection: whereas several english datasets with annotations of framing have been released (see section 2), for any language other than english, to our best knowledge, there is still no annotated dataset to date. addressing the lack of investigation on the linguistic aspects of framing, we bring together both shallow topical cues and in-depth linguistic devices to detect framing strategies in german newspaper articles on european refugee crisis. the novelty of our work is the investigation of how subtle semantic and pragmatic features contribute to framing a message: in theoretical linguistics, researchers have discovered a variety of subtle linguistic devices that play a fundamental role in expressing the speaker's attitude or leading the addressees to integrate information into their belief systems in a certain way. consider the highlighted expressions in example (1): the expression nicht einmal 'not even' reinforces the author's critical attitude to the ruling parties by conveying that the meaning of family in asylum cases should be the most basic knowledge they ought to have, but actually they do not. besides, by using the modal particle ja, which does not have an english equivalence but can be loosely paraphrased to 'as we all know', the author subtly renders his opinion as already being a consensus of all people (even if this might not be true), covertly increasing its credibility. die drei parteien wissen ja nicht einmal, was im falle der flüchtlinge [...] unter familie zu verstehen ist. 'the three (ruling) parties do not even understand what family means in the case of the refugees -as we all know.' (source: frankfurter allgemeine zeitung) linguistic devices of such kind do not contribute to the topical content of the text, but frame the utterance by adding rhetorical flavors that reinforces specific stances of the author. in what follows, we refer to such framing effect as rhetorical framing. building upon linguistic theories and insights from political science research on framing, we operationalize a set of deep semantic and pragmatic features that are relevant to rhetorical framing, and apply them to data-driven framing detection in a large-scale dataset with 8 million tokens. our study makes the following contributions: (a) at the theoretical level, we propose a variety of deep semantic and pragmatic features relevant to framing, and illustrate their subtle yet powerful role in framing with a quantitative study. our proposed linguistic features provide novel insights towards a deeper understanding of framing, and can also inform future work on creating annotation schemata for framing. (b) at the methodological level, we release a heuristic-based automated annotation pipeline for the proposed linguistic features. 1 ## related work the release of the policy frames codebook (pfc; @xcite has facilitated the task of creating datasets and building models for automated framing detection. pfc proposes 14 topic-oriented frame categories that can be applied to any policy issue, e.g., economic frames, morality frames, or security and defense frames. this has provided a convenient basis for building annotation and classification. since then, researchers have published several english-language datasets with manual an1 the dataset used in our study was purchased from the publishers. due to their copyright regulations, the dataset is restricted to project-internal usage and unfortunately cannot be distributed to third parties. but all code and lexical resources resulting from this paper are publicly available at: https:// github.com/qi-yu/topical-and-rhetorical-framing notation of frames using the taxonomy of pfc or a similar topic-oriented fashion (e.g., @xcite @xcite . owing to the pfc and these publicly available datasets, previous nlp work on framing has mainly focused on identifying topical framing. the recent semeval 2023 shared task on framing detection foot_0 also adopts this topic-oriented setting. the approaches applied in previous studies range from fully unsupervised to fully supervised methods: @xcite and @xcite rely on unsupervised topic models. @xcite and @xcite compile framing vocabularies to measure the prevalence of different frames. using a fully supervised fashion, @xcite build naïve bayes classifier using theoretically derived linguistic features. more recent work has been leveraging powers of nns, e.g., @xcite (lstm) and @xcite . especially, transformer-based language models have been widely used in the last years @xcite @xcite @xcite @xcite . however, studies using topic-oriented taxonomies of framing tend to oversimplify the concept of framing as a mere matter of topic coverage. this is insufficient for a deep understanding of framing. in political science, it is widely pointed out that framing is a multi-faceted phenomenon: it includes not only the information sender's intention of reinforcing specific topics, but also the facet of how the frames in a communication process affect the individual's thought @xcite . for the second facet, the usage of subtle linguistic devices can play a crucial role. especially, certain pragmatic markers have an effect in manipulating mutual assumptions or facilitate processes of pragmatic inferences @xcite . in nlp, the impact of individual linguistic components in framing remains understudied with only a few exceptions: @xcite utilize various semantic cues (factive verb, assertive word, entailment and hedging) to classify framing, but the authors do not provide discussion on what textual or rhetorical effect these cues have in framing a message. @xcite and @xcite inspect the usage of deon-tic modal verbs (e.g., 'should', 'need') in calling for actions, assigning blames and making moral arguments. however, their argued importance of modal verbs only applies to texts with a primary function of calling to actions, but does not necessarily generate to all text types. @xcite investigate the usage of agentless passives (e.g. using 'he was killed' instead of 'he was killed by police') in removing blames. @xcite shows that iterative adverbs such as 'again' can compose systematic framing strategies by evoking attitudinal subtexts via presuppositions. we follow this strand of work on linguistically informed framing detection, and quantitatively investigate the effect of a wider variety of semantic and pragmatic cues. we aim at extending the existing knowledge on the linguistic composition of framing. ## data we focus on a dataset comprising of articles about the "european refugee crisis" published between 2014 to 2018 by the three most circulated newspapers in germany: bild, frankfurter allgemeine zeitung (faz), and süddeutsche zeitung (sz). all three are nationwide daily newspapers, and they build a balanced sample of different styles (tabloid vs. quality) and political orientations. from each newspaper, we first collected articles with at least one match of the following quasisynonyms of 'refugee' (including their inflected forms): {flüchtling, geflüchtete, migrant, asylant, asylwerber, asylbewerber}. we then removed articles that were: 1) duplicated, 2) from non-political sections such as sport, and 3) with a ratio of the 'refugee'-synonyms lower than 0.01. criterion 3) was experimentally defined: it allowed us to remove most articles that mention the european refugee crisis only as a side-topic. as we do not have any annotation of frames available for our dataset -moreover, it is prohibitive to conduct such annotation considering the enormous labor cost, we use a set of theoretically derived topical and rhetorical features to conduct a data-driven exploratory framing analysis on the document level. the proposed rhetorical features (section 4.2) also aim to fill the research gap in investigating the linguistic properties of framing. the features and their relevance to framing are described below. ## results and discussion figure 2 shows the within-source frequency of each feature, computed as the count of a feature divided by the total token amount of a source. it can be observed that the rhetorical framing features are extremely sparse, especially the modal particles. 5 yet, the logistic regression analysis reveals interesting contrast between the usage of the features in each newspaper. figure 3 shows the z-values of all significant features in predicting each source. the detailed results are provided in appendix a. in what follows, we summarize the major findings from the logistic regression analysis. ## conclusion and outlook previous research in framing detection has focused heavily on topical framing, leaving the effect of individual linguistic devices in framing understudied. addressing this weakness, we theoretically derived a set of in-depth semantic and pragmatic features relevant to framing, and implemented an automatic annotation pipeline for identifying them. combining them with shallow topical framing cues enabled us to identify deeper differences in framing strategies employed by different german newspapers in the discourse of the european refugee crisis. the advantage of our approach is its linguistic depth and explainability: to our best knowledge, all the proposed features have still not been studied in respect of framing in a large-scale fashion. our work contributes to both nlp and social sciences by extending the knowledge of linguistic aspects of framing. for future nlp work on framing detection, this work has two indications: first, framing detection should not be restricted to topical frames. many linguistic devices also play crucial roles in framing by affecting how a message should be received by individuals. second, though handcrafted feature sets have multiple restrictions, a more in-depth framing detection can still benefit from consciously incorporating theoretically derived features. as shown by our study, the distribution of many important linguistic devices could be extremely sparse, whose effects might thus be challenging for nn-based algorithms to capture. this work is not without limitations. first, the various types of modal particles involved in this work do not exist in all languages, and their taxonomy in different languages can vary from the one applied here. our automated annotation pipeline is also german-specific by now. however, other rhetorical framing devices and their effects discussed in this work are language-independent and can thus be applied to framing analysis for other languages. second, this work only covers a limited range of linguistic features: due to the lack of existing tools or annotated datasets, we adopt a rule-based approach for the automated feature identification. some relevant cues for the rhetorical framing features are left out because their disambiguation is highly context-dependent and difficult to realize with rules. future work will consider using our automated annotation pipeline as a weak-labeling assistance and creating datasets for detecting the linguistic features with supervised methods. moreover, for modal particles and presupposition triggers, a more fine-grained analysis direction would be to identify the actors involved in sentences containing these devices, and detect frames using discourse network analysis on the actors (following van @xcite . despite these limitations, we hope that our initial work on rhetorical framing strategies will facilitate future work on investigating the deeper linguistic dimensions of framing.