Large Scale Narrative Messaging around Climate Change: A Cross-Cultural Comparison (2024)

Abstract: In this study, we explore the use of Large Language Models (LLMs) such as GPT-4 to extract and analyze the latent narrative messaging in climate change-related news articles from North American and Chinese media. By defining “narrative messaging” as the intrinsic moral or lesson of a story, we apply our model to a dataset of approximately 15,000 news articles in English and Mandarin, categorized by climate-related topics and ideological groupings. Our findings reveal distinct differences in the narrative values emphasized by different cultural and ideological contexts, with North American sources often focusing on individualistic and crisis-driven themes, while Chinese sources emphasize developmental and cooperative narratives. This work demonstrates the potential of LLMs in understanding and influencing climate communication, offering new insights into the collective belief systems that shape public discourse on climate change across different cultures.

## introduction understanding the stories we tell is a key priority for those engaged in climate discourse. stories serve as a fundamental method through which people exchange information and forge shared understandings about cause and effect in our world, essentially explaining "why things occur" @xcite . studies indicate that storytelling can be an effective means of overcoming resistance to new ideas and changing people's intentions to act @xcite @xcite . consequently, there is significant interest among climate advocates in leveraging the power and influence of narrative to shift public perspectives @xcite . in this paper, we propose a method for surfacing the latent narrative messaging of news stories related to climate change. narrative messaging refers to an overarching, higher-level message that a given story conveys to its readers, one that may be more or less explicit in the body of the story. narrative messaging is thus akin to broader narratological concepts such as "schemas" @xcite , "archetypes" @xcite , "frames" @xcite , and "meta-narratives" @xcite . despite addressing narratives at varying levels of abstraction, these models converge on a fundamental premise: stories inherently share common elements, and their selection is orchestrated by higher-level schemas or messages that shape the narrative's construction and interpretation. for our purposes here, we define narrative messaging as consisting of a story's "moral" or "lesson," i.e. an intrinsic message that readers are intended to take away that transcends the specific details of the story. for example, in a fable such as "the lion and the mouse," the moral / message of the story is "a kindness is never wasted." in a news article about climate change that focuses on policy disputes, the moral / message might be "political compromise is important for finding solutions." while we typically think of story morals as reserved only for short didactic fiction (such as fables), narratologists have long argued that all narratives have implicit value-driven schemas that govern how they are told @xcite . as a narrative message, a story moral focuses on the values and intentions of the storyteller. the moral or lesson of a story is in some sense an answer to the question, "why is this person telling me this story?" rather than focus on the specific content of the story, attention to story morals focuses on a more general lesson to be learned ("economic interests and energy security concerns often hinder global consensus on phasing out fossil fuels"). surfacing such latent narrative values at large scale can facilitate the process of understanding collective beliefs around particular societal concerns, their differences across cultures, and any meaningful changes over time. in our work, we leverage the affordances of large language models (llms) like gpt-4 to extract a series of narrative features associated with a given news story up to and including the story's central "moral" or "lesson" (fig. 1 ). such features include the identification of the main agent of the story, the central topic of the story, any antagonist or negative agent, the story's overall valence, along with a free-form moral that is also rendered as a single keyword or phrase. llms present a promising avenue for automating the labeling of story morals. despite ongoing challenges with hallucinations in llms @xcite , their ability to infer underlying meanings-akin to deriving morals from narratives that are often implicit rather than explicitly stated-aligns well with their capabilities. furthermore, the ubiquity of narratives and narrative-like moral statements on the internet suggests that these concepts are likely well-represented in llm training datasets. however, this also necessitates a critical evaluation of potential cultural biases embedded within these models. we proceed first with a review of prior work related to our topic, particularly with respect to prior work on climate communication. we then introduce and validate our method using a combination of automated metrics and human annotation. finally, we apply our method to the study of a collection of ca. 15,000 news articles written in english and mandarin that are subsetted by different climate-related topics and different ideological groupings (state / offshore, left / right, see ## prior work in the field of "environmental communication," the notion of "story morals," as investigated in this study, closely aligns with the concept of "framing," which has been extensively examined in prior research. framing, sometimes also known as schemas, involves highlighting certain aspects of perceived reality in communicative texts to promote specific problem definitions, causal interpretations, moral evaluations, and recommended solutions @xcite . framing's significance is particularly evident in discussions on environmental issues, where climate change represents a "super wicked problem" characterized by the urgent need for action, yet hindered by delayed impacts and insufficient institutional efforts @xcite @xcite . consequently, effective framing is essential to bridge the gap between awareness and action in environmental protection @xcite @xcite . a substantial body of literature has explored various frames in climate discourse, such as "social progress," "scientific uncertainty," and "conflict" @xcite @xcite , covering different periods and geographic regions @xcite . some studies compare climate issue framing in various nations @xcite @xcite ; while others perform temporal analyses correlating media coverage with significant climatic and political events, such as the cop and kyoto protocol @xcite @xcite . traditionally, these studies have relied on manual coding methodologies, where coders are trained to identify specific elements in articles, such as scientific controversies, typically resulting in a dataset comprising a few hundred articles. recently, however, automated methods like topic modeling have been adopted in climate framing research to enhance data analysis efficiency @xcite . our work can be seen as a further extension of such automated approaches, but with a novel focus on the values of narrative messaging by leveraging the affordances of llms. within the nlp community, the analysis of narrative understanding has gained significant interest @xcite . this research encompasses the aim of narrative detection and understanding, across varied contexts, such as literature, social media, and health-care related communication @xcite . recent research in this area has also begun exploring the idea of "collective narratives" which involves synthesizing smaller narrative elements (such as tweets, blog entries, or news articles) into overarching narrative frameworks @xcite . the research collaborative climate change ai has been an important early mover in bringing together the ml and nlp fields with climate change concerns @xcite . our work builds off this prior work by bringing together the approaches and theories of computational narrative understanding towards the goal of studying climate-related communication in different cultural contexts. instead of applying manually labeled codes to smaller collections, we show how gpt-4 can help surface intrinsic narrative schemas related to the implicit values driving large volumes of news production and that those schemas align with human judgments. in addition to scaling up our understanding of climate-related communication, our approach also shifts the focus from content-related questions (i.e. "what happened") towards more value-driven questions (i.e. "why is this being told?"). doing so, we argue, can help surface important insights into the collective and often latent belief systems that govern what stories get told and how. ## conclusion our study illustrates a workflow that can be applied to understand the narrative messaging of values around climate change across different cultural contexts. we show the robust validity of large language models like gpt-4 to derive highlevel conceptual information about narratives in strongly different cultural and linguistic contexts. in particular, we surface key "values" associated with climate-related news, with chinese media focusing on a more "developmental" approach compared to a more "adaptive" approach on the north american side. such workflows will be an important dimension towards scaling up our understanding of climate communication. while we focus on surfacing implicit narrative values around climate change from the bottom-up, researchers can also use our method to test more specific hypotheses and content-related "frames" determined in advance. as we discuss in the limitations section, more work is necessary to better understand the biases or norms implicit in llms. nevertheless, we believe llms are going to be an important tool in understanding, interpreting, and influencing climate communication moving forward.