Framing in the Presence of Supporting Data: A Case Study in U . S . Economic News (2024)

Abstract: The mainstream media has much leeway in what it chooses to cover and how it covers it. These choices have real-world consequences on what people know and their subsequent behaviors. However, the lack of objective measures to evaluate editorial choices makes research in this area particularly difficult. In this paper, we argue that there are newsworthy topics where objective measures exist in the form of supporting data and propose a computational framework to analyze editorial choices in this setup. We focus on the economy because the reporting of economic indicators presents us with a relatively easy way to determine both the selection and framing of various publications. Their values provide a ground truth of how the economy is doing relative to how the publications choose to cover it. To do this, we define frame prediction as a set of interdependent tasks. At the article level, we learn to identify the reported stance towards the general state of the economy. Then, for every numerical quantity reported in the article, we learn to identify whether it corresponds to an economic indicator and whether it is being reported in a positive or negative way. To perform our analysis, we track six American publishers and each article that appeared in the top 10 slots of their landing page between 2015 and 2023.

## introduction the mainstream media has much leeway in what it chooses to cover and how it covers it. should the top article at any given point in time be about an international incident or the state of the economy? conditional on choosing to cover the economy, should the article focus on the indicator that is up or the one that is down? should it be optimistic or pessimistic towards the future? these choices have real-world consequences affecting outcomes such as voting behavior @xcite , gun purchases @xcite , and attitudes towards immi-figure 1 : the frame prediction framework grants @xcite . most selection and framing decisions lack objective measures, making research on this topic difficult. in this paper, we make the observation that there are a select number of newsworthy topics where objective measures do exist. for example, there is a finite set of economic indicators that experts use to define the state of the economy, such as changes in non-farm payroll and gross domestic product. another example is crime, which has well-established data points maintained by the fbi and local jurisdictions. building a system capable of tracking a large number of publications, identifying which indicators are reported and how they are being reported has enormous research potential. such a system would enable us to create three views of potential bias: how a publication is covering a topic over time, how their coverage differs from other publications, and how it compares to the accepted ground truth of experts in the field. following this rationale, we propose a computational framework to predict frames in the presence of supporting data. for a given article, our goal is to automatically identify how the general topic is being portrayed, which indicators are being reported to support this view, and how each of these indicators are being presented. we focus specif-ically on news articles about the u.s. economy. however, we argue that this framework could be adapted to other domains where numerical indicators are used, including reports on crime, climate change, and public opinion. most previous computational framing analysis approaches have conceptualized and interpreted frames as high-level relevant topics and themes @xcite . for example, @xcite proposed 15 broad dimensions that aim to capture ways in which policy issues are discussed in news articles. the dimensions include themes such as "economic" and "public opinion". these broad dimensions fail to capture a frame's nuances. for example, the "economic frame" focuses on anything related to the economy, which is insufficient to answer the question of how different aspects of the economy are being presented. to address this problem, we decompose economic frames into a set of interdependent tasks (see fig. 1 ). at the article level, we identify the relevance of the article to the economy, the type of economic information it covers (e.g., macro economic, industry-specific, firm-specific), and two measures of general economic sentiment: the state of the economy (e.g., good, fair, poor), and the direction in which the economy is heading (e.g., better, same, worse). then, for each numerical quantity reported in the article, we predict whether it corresponds to an economic indicator (e.g., market numbers) and its reported polarity (e.g., "the s&p 500 fell 3%" has a negative polarity). this decomposition captures three levels: whether the economy is being discussed, which economic indicators are highlighted, and the framing of those indicators and the economy as a whole. this way, we can go beyond analyzing topic selection by additionally measuring how a publication's reporting differs from the objective reality at that time, how it evolves across time, and how it compares to other publications. we present a computational model that predicts each proposed frame component jointly. it utilizes the inter-dependence between the sub-tasks (e.g., polarity of quantities and polarity of the article they appear in) to improve upon results obtained by independent classifiers trained to make predictions individually with limited supervision. we make the following contributions: (1) we propose a computational framework to model framing in the presence of supporting data and apply it to news about the u.s. economy. (2) we col-lect a novel dataset of landing page news articles published by a set of major u.s. news outlets between 2015 and 2023, and provide high-quality annotations of our proposed frame components for a small subset of examples. (3) we propose an automated method to detect each of the proposed frame components under low-supervision settings. (4) we demonstrate that our framework can be used to track when and how often (selection), as well as how (framing) different news outlets report on specific aspects of the economy. we compare these reports to the ground truth for two example indicators: job numbers and prices. all of our code and data has been released to the community * . ## background and related work in this section, we cover the background on selection and framing, explain economic indicators, and discuss previous approaches to analyze framing. selection and framing national news publications have a massive choice-set of topics they could cover on any given day. though it may seem like the publications are reacting to a defined set of events that recently happened, outside of a few major events, they choose what the news is on any given day. in other words, the news in mainstream publications does not present an objective view of current affairs. rather, it is influenced by the selection made by media professionals @xcite . selection is driven by what researchers have called news values @xcite @xcite , which determine whether a story is newsworthy. while there is variation in the taxonomies that have been proposed to determine newsworthiness, the most recent ones include factors such as: the power of the elite (stories concerning powerful entities), bad and good news (stories with particular emotional overtones), and magnitude (stories that are perceived as significant for a wide audience) @xcite . more interestingly, news values vary from publication to publication. a study of the run-up to the 2022 election showed that the new york times and washington post averaged just two overlapping topics each morning in their printed editions out of about six articles @xcite . conditional on selecting a topic, there is additional leeway on how to cover it, as there is a wide-range of facts and opinions that could spin the takeaway of the readers. this phenomenon, referred to as framing, has been widely studied in communication studies. while there is not an agreed upon definition of framing, recent surveys @xcite have identified the following prevalent definitions: equivalence framing (presenting the same exact information in different ways) @xcite , emphasis framing (highlighting specific aspects of an event to promote a particular interpretation) @xcite , and story framing (leveraging established narratives to convey information) @xcite . when it comes to the economy, a publication could present a rosy or terrible version of the economy. they can do so without lying by simply picking the right economic indicator with the right perspective. for example, there are months where the value of the non-farm payroll is above average, but below the market expectation: either perspective is technically true. but, with a relatively finite set of indicators, and an established way for economists to judge them, there is still a clear ground truth to compare if any given indicator with any given perspective represents the consensus understanding of the economy at that time. economic indicators understanding the health and wealth of the u.s. economy is critical for investment and growth. most modern economic indicators have been tracked for decades, codified in the by the mid-20th century. examples include: jobs numbers (e.g., non-farm payroll), prices (e.g., consumer price index), and macro economic (e.g., gross domestic product). with the exception of market prices (e.g., s&p500), which are a reflection of the underlying assets, the core indicators are tracked by a few government agencies (e.g, the bureau of economic analysis for gdp, spending, income, trade, and the bureau of labor statistics for unemployment, prices, productivity), which conduct regular, large scale polling. these economic indicators inform the public on the health of the economy. the job numbers are front page news when they are released at 8:30 am et on the first friday of each month. the state of the markets is constantly covered in the news. indicators such as the consumer price index pop up whenever the media finds them most important, interesting, and engaging. most people have limited views of the economy outside of their close social circle, so these values are the key to their under-standing of the health of the economy as a whole. and, traditionally, the health of the economy has tracked very closely with the approval of incumbent political leaders @xcite . framing of economic news this paper builds off of an extensive literature that has not just identified the quantity of economic coverage, but framing of that coverage. most of this work is geared towards matching the sentiment of articles about the economy to both consumer sentiment and economic indicators, tracking both historical value and predicting future ones @xcite @xcite @xcite . while the methods continue to evolve from simple keywords to bert or llm-based predictions, they are focused on article-level sentiment, aggregated to states or the u.s. they also study a mix of questions about economic impact that benefit from a long time-series of sentiment, granular in both time and geography, regarding questions as diverse as the impact of bubbles on industries @xcite , economic shocks to economic decisions @xcite , to the relationship between news and perception of the economy @xcite . computational framing analysis scholars have increasingly adopted computational approaches to study framing, allowing studies to scale to large media repositories. in most cases, researchers adopt unsupervised techniques such as topic modeling to identify latent themes @xcite @xcite . however, topic models are limited in their ability to capture framing nuances. by defining topics as simple distributions over words, they lack the semantic and discursive conceptualization needed to answer how issues and events are being presented @xcite . there is also a body of work leveraging supervised learning @xcite @xcite @xcite and lexicon expansion techniques @xcite ) to analyze framing. for this to be feasible, authors must define a concrete taxonomy of which frames are relevant and how they are represented in data. most of these approaches rely on the taxonomy proposed by @xcite . these dimensions correspond to broad themes such as "economic", and "public opinion". while these themes have shown to be useful to model a select set of contentious political issues like abortion and immigration, they are too broad to capture how the economy and other data-driven topics are being presented in the media. ## data in this section, we describe our data collection process, annotation schema, annotation guidelines, and quality assurance process. we also present statistics for our resulting dataset. ## model obtaining high-quality annotations for our task is time-consuming and requires considerable domain expertise. for this reason, one of the main challenges that we face in predicting economic frames is a small amount of supervision. to circumvent this constraint we combine two modeling strategies: (1) exploiting the decomposition of frame prediction into a set of modular, inter-dependent sub-tasks, and (2) leveraging pre-training strategies using our large set of in-domain unlabeled examples (all 199,066 economic articles collected). exploiting inter-dependencies with srl statistical relational learning (srl) methods attempt to model a joint distribution over relational, interdependent data (richardson and domingos, 2006; bach et al., 2017; pacheco and goldwasser, 2021). these methods have proven particularly effective in tasks where contextualizing information and interdependent decisions can compensate for a low number of annotated examples (deng and wiebe, 2015; roy et al., 2021; pacheco et al., 2022a). to model framing, we take advantage of the relations between the values of the frame components for a given article and implement a relational model using probabilistic soft logic (psl) @xcite . dependencies in psl are expressed using weighted logical rules of the form: w r : p 1 ∧ ... ∧ p n-1 → p n , where w r indicates the importance of the rule in the model, and can be learned from data. predicates p i correspond to decisions and observations. rules are then compiled into a hinge-loss markov random field and weights are learned using maximum likelihood estimation. we consider the following rules: priors. we explicitly model prior probabilities for all article-level (type, economic conditions and economic direction) and quantity-level (type, macro-indicator, and sentiment) components. these priors are derived from supervised classifiers trained on our labeled dataset. rule r 1 and r 2 are modeled as hard constraints, and are always enforced because they are designed to force predictions to follow the frame annotation structure. rules r 3-5 are modeled as soft constraints, and their weights are learned through psl. this way, we allow for predictions that do not conform to the templates to be active. enhancing psl priors with pre-trained language models pre-trained language models (lms) are one of the most effective techniques for acquiring knowledge from unlabeled text data @xcite . domain-adaptive pre-training (dapt) further enriches lms with in-domain data @xcite . in both of these pretraining stages, learning is conducted by randomly masking words in a large dataset and training the the lm to predict these words. to take advantage of these strategies, we use pre-trained roberta and perform dapt with our large unlabeled dataset (all 199,066 economic articles collected). we use this lm to fine-tune the classifiers that are used as priors in psl. this strategy has been repeatedly used to combine psl with strong classifiers, with consistent success @xcite @xcite . ## experimental evaluation in this section, we outline our experimental settings and present results for our frame prediction model. we also include an example of the type of framing analysis that can be done using our model predictions over the complete dataset. ## applicability to other domains we emphasize that the main contribution of this paper is not centered around performance gains of our relational model over the baseline models, but rather the operationalization of a challenging task: analyzing framing at a scale for domains with supporting numerical data, such as the economy. while our solution does not apply to all framing scenarios, it applies to a substantial number. for example, we envision this framework being useful for analyzing news about crime (by tracking data points maintained by the fbi and local jurisdictions), climate change (by tracking weather statistics, global avg. temperature, emissions statistics, etc.) and public-opinion (by tracking polling numbers, political betting market numbers, etc.). ## conclusion and future work in this paper, we presented a novel framework for modeling framing in news articles in the presence of supporting data. we focused our analysis on news about the u.s. economy, and proposed an annotation schema to identify framing at three levels of abstraction: the general framing of the economic conditions, the economic indicators that are highlighted, and the framing of each of those indicators. we showed that we can predict the components of our schema with relatively good performance, even in very low supervision settings. finally, we demonstrated how our framework can be used to perform a large scale exploration of the frame choices for a given economic indicator. in our analysis, we showed that different publications cover economics using different indicators, for example the nyt covers jobs numbers less than normal during a strong jobs recovery, instead switching to more coverage rising prices. and, while all three publications responded to rising inflation, they each used price numbers differently when covering the economy, with the nyt employing price numbers more frequently than the washington post, or the wall street journal. going forward, we have two main points of focus. first, we are working on expanding our schema to capture a larger set of economic aspects. while we focused our analysis on macro-economic indicators, there are other indicators that we could track to get a more holistic view of the economy. for example, for governmental indicators, we could track types of expenditures (e.g. social security), revenue (e.g. taxes), as well as debt and deficit. additionally, we want to perform a large-scale analysis of economic news framing. in this paper, we presented a brief demonstration of how our framework can be used to analyze shifts in framing across time for different publications. next, we want to perform a similar analysis for a larger set of economic indicators and publications. we note that to perform such an analysis we must scale up our annotations and improve our model predictions.