Media Attitude Detection via Framing Analysis with Events and their Relations (2024)

Abstract: Framing is used to present some selective aspects of an issue and making them more salient, which aims to promote certain values, interpretations, or solutions (Entman, 1993). This study investigates the nuances of media framing on public perception and understanding by examining how events are presented within news articles. Unlike previous research that primarily focused on word choice as a framing device, this work explores the comprehensive narrative construction through events and their relations. Our method integrates event extraction, cross-document event coreference, and causal relationship mapping among events to extract framing devices employed by media to assess their role in framing the narrative. We evaluate our approach with a media attitude detection task and show that the use of event mentions, event cluster descriptors, and their causal relations effectively captures the subtle nuances of framing, thereby providing deeper insights into the attitudes conveyed by news articles. The experimental results show the framing device models surpass the baseline models and offers a more detailed and explainable analysis of media framing effects. We make the source code and dataset publicly available.

## introduction media framing significantly shapes public perception and opinion by highlighting or downplaying certain aspects of events @xcite . in news articles reporting putin's 2024 election win worldwide, supportive articles emphasize the legitimacy of the election and present it as the reflection of the majority of russian voters' will. these articles argue that putin has brought economic growth and a strong international presence to russia. in contrast, skeptical articles highlight incidents of electoral fraud and manipulation and mention deaths of putin's political opponents and ukrainian war. in this paper, we conduct framing analysis on highly contentious international news articles by strategically extracting and encoding event information to detect the media attitude reflected in the news articles towards the main event. framing analysis introduces rigorous and systematic methods for studying media content. this approach enriches research in communication, journalism, and social sciences by providing tools to dissect and interpret narratives within news articles @xcite . identifying and understanding different perspectives through framing analysis can encourage more balanced and fair reporting. previous work only focused on the extraction for frames that are more aligned with topics and the choice of words out of many framing devices. they answered "what is presented" (the issue) instead of "how it is presented" (framing) @xcite . narratives in media framing can be constructed by linking multiple events together @xcite . this process involves creating a broader storyline or context that connects individual events to convey a more comprehensive theme. by analyzing how events are used to frame a topic, we can provide a deeper understanding of the attitude of a news article. this approach goes beyond equating frames with topics and text span based framing detection that are common in computational framing research @xcite , offering deeper insights into narrative construction and media attitudes. we employ a pipeline approach for detecting media attitudes through framing analysis using events and their relationships. figure 1 illustrates the full pipeline for the media attitude detection task. the process begins with the collection of news articles on various topics. for each article, we extract all events mentioned from its main content. then, we cluster coreferent events from all articles of the same topic and generate a shared event descriptor for each cluster. this descriptor represents all event mentions within that cluster across the documents, ensuring consistency and clarity in event representation. given the event mentions, we extract causal relations among the events for each article, mapping event mentions to their respective cluster descriptors. we then evaluate the effectiveness of encoding coreferent event descriptors, encodings of event mentions, and causal relationships as framing devices. this is achieved by detecting the attitudes of the news articles by fine-tuning small language models and prompting large language models (llms). our results demonstrate that these framing devices are not only effective in capturing framing nuances but also provide explainable insights into how media frames the narratives. the experimental results indicate that all three methods of encoding framing devices outperform the baseline models. specifically, in the case of gpt-4o, device encodings improve over the baseline encoding by ranging from 13 to 28 points. the best-performing models are the fine-tuned models incorporating framing devices. we summarize the major contributions of this paper are as follows: framing device concepulization we address three open questions in computational framing posed by ali and hassan (2022): first, we extend beyond the traditional focus on word choice as a framing device. our computational methods capture events and their causal relations, facilitating a more nuanced exploration of frames. second, we employ the cross-document event coreference technique to capture semantic relations within a single document and how these relations are informed by those in other documents. this approach provides a broader and more integrated understanding of frames across multiple documents. third, we broaden the scope of framing devices to include the selection or omission of events, as well as the causal relations among events, in addition to the conventional focus on word choice. this allows for a more comprehensive analysis of how framing is achieved through different mechanisms. new dataset for media attitude detection we provide a robust dataset annotated with media attitudes. it contains over 1,600 articles from three topics. this dataset serves as a resource for further research in computational framing analysis and related fields. better performing models with framing devices our models significantly outperform the baselines, showing an improvement over the baseline encoding. they achieve higher overall accuracy, demonstrating that the three framing devices are effective for compressing information in news texts. additionally, our input length is significantly shorter than that of the baseline models, resulting in reduced computing time and more efficiency. moreover, our proposed encodings, namely the event descriptors, event mentions, and causal relationships enhance the explainability of the models' results. the rest of the paper is organized as follows. in §2, we discuss related work on computational framing in terms of conceptualization and computational methods. §3 describes the media attitude detection task and our adoption of three framing devices in encoding. §4 describes the data collection and data annotation process. §5 describes event detection, cdec, event descriptor generation and event causal relationship extraction we adopted in the pipeline. §6 describes the setup of the four media attitude detection models, presents the results and analysis. finally we conclude in §7. ## related work the policy frames codebook @xcite is a codebook that was used by @xcite to develop the media frames corpus. they established 15 framing dimension categories that tend to resemble topics more than actual frames. the gun violence frame corpus @xcite annotated news headlines but did not fully elaborate on the framing aspects defined according to @xcite . instead, it used keyword clues to categorize the framing dimensions. computational methods have been increasingly adopted in framing analysis. @xcite employed lda topic modeling to explore frames, but this approach is more about theme, lacking the ability to capture framing nuances such as causal interpretations. @xcite introduced "supervised hierarchical latent dirichlet allocation (shlda)," which generates a hierarchy of topics with first-level nodes as agendas and second-level nodes as frames. this subtopic framing approach does not align with framing conceptualizations @xcite . additionally, @xcite applied k-means clustering on a dataset, using word frequencies as frame features in their cluster analyses. however, it fails to capture how events are interconnected through cause-and-effect relationships, which are essential for understanding the underlying narrative and motivations presented in the text. furthermore, @xcite employed fine-tuned roberta models to create classifiers for identifying frames in news articles. although their approach proved effective, the conceptualization of framing and the categorization of their dataset could be improved by incorporating more detailed and nuanced framing aspects. typically, computational methods tend to only partially capture the complexity of framing, often relying solely on single framing devices such as word usage. this approach overlooks the multiple dimensions and layers of framing analysis. in a related study, @xcite extracted events and assessed their impact on the stance of news articles by identifying events indicative of ideology. however, the authors noted they did not further explore how these events relate to each other to convey ideology. for instance, their extraction method could detect a right-leaning event alongside several left-leaning events that oppose it, without explaining the interaction between these events. in narrative framing research, the perception to narratives is influenced by event temporal structure and shared event scripts @xcite . furthermore, narratives act as tools for social construction, shaping public opinion and policy by framing events in particular ways @xcite . ## dataset we describe the data collection and annotation process for creating the media attitude detection dataset. ## data preparation to evaluate the media attitude detection via framing analysis, we prepare the dataset by automatically creating encodings of different framing devices from extracted events and relations. ## experiments we evaluate our proposed framing device encodings under different model settings. we experiment on the media attitude detection task with model inputs encoded from different framing devices, and compare them to the baseline setting with raw texts as the input. for each topic, we randomly select 70% articles from the dataset for training and the remaining 30% for testing. we use accuracy as the primary evaluation metric. ## conclusion framing analysis helps uncover the underlying attitude that news outlets may have. by analyzing how events are framed, we can better understand the intentions and viewpoints of the media sources, providing insight into how information is presented to the public. our methods extend beyond words by conceptualizing framing devices and integrating event extraction, cdec, and causal relationship extraction. our media attitude detection through framing analysis demonstrates that all three framing devices effectively captures the subtle nuances of framing, providing deeper insights into the attitudes conveyed by news articles. our results indicate that training with framing devices achieves competitive results on some topics and improves training efficiency by utilizing much shorter and more explainable model inputs. llms incorporating framing device information significantly outperform the baseline and achieve results comparable to those of supervised models. we hope that our analysis and improvement on the media attitude detection task can facilitate further developments by future researchers.