Towards Designing Social Interventions for Online Climate
Change Denialism Discussions. In .(https://doi.org/XXXXXXX.XXXXXXX)
1 INTRODUCTION
Climate Change Denial Post 1: This is actually two horrible scandals rolled into one. Lyme disease has been suppressed and ignored for decades by the conventional medical complex in the US and Canada at the behest of the CDC most likely because chances are it was designed as a bioweapon at the Plum
Island facility just off the coast of Old Lyme, CT [Connecticut] where it was seen for the first time, hence the name. ...Ticks are very hardy, and can live in many habitats, but in the US it’s the eastern seaboard,
mid-west, and west coast that are the most infiltrated, not the southern US where the climate is naturally
warmer. So this claim that ticks can’t live in colder climates and global warming is allowing them to expand their territory is entirely bogus, but suits two bogus agendas: “climate change” and the medical establishment’s horrendous but deliberate handling of Lyme disease. I have to give them credit for trying to roll all these criminal actions into one though as it probably fools most people. ( r/climateskeptics )
Climate Change Denial Post 2: They [The governments] are politicizing any fire to be about climate change now. They did it recently with the Amazon.
Publication rights licensed to ACM. Manuscript submitted to ACM CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
they are taking completely normal, natural occurrences, that happen every year for the past 1000 years,
and rebranding them as proof of climate change. ( r/conspiracy )
Climate Change Denial Post 3: Don’t you feel a bit silly thinking a trace gas in the atmosphere could cause the climate to change and cause catastrophic events, none of which have ever happened. No sea level rise . only 1.3 degrees warming in 140 years no increase in hurricanes, floods, droughts or wildfires, dramatic reduction in deaths caused by the environment, no climate refugees, no empirical evidence of cause and effect at all. ( r/climateskeptics )
These examples are only three of many conspiracy theories (CT) about how climate change is formed and how people should engage with climate change. Though large amounts of research have argued how the increasing amount of carbon dioxide is impacting global temperature and how the sea level has been increasing for the past
decade, there has been continued disbelief of scientific evidence, especially in communities that are involved with climate change denialism. With the rise in the attention for conspiracy theories (CT), it has become important to explore mechanisms through which CT believers could be dissuaded from CT beliefs. Key characteristics of CT include 1) highlighting secretive actions of certain groups of people to achieve some end result 2) claiming the existence of unethical behavior to achieve power, control, or other benefits 3) showing excessive distrust in official sources of information 4)
demonstrating selectiveness of evidence or misinterpretation of evidence and 5) postulating that contradictory evidence is people’s plot of the conspiracy itself . In the aforementioned examples from r/climateskeptics
and r/conspiracy, the first example illustrates how climate change denialism believers consider that the claim of global warming is to cover up the occurrence of Lyme disease, demonstrating their belief in unethical behavior made by the medical establishments to achieve secret agenda. This corresponds to the first and second characteristics of CT. In the second example, climate change denialism believers showed a clear distrust in the government’s claims and reports,
which is related to the third characteristic of CT. The third example shows that climate change denialism believers
cherry-picked evidence to make their claim and completely dismissed evidence about the rise in sea level from existing
research, corresponding to the fourth characteristic of CT. In addition, their dismissiveness of the evidence of rise in temperature and natural disasters shows their tendency to ignore any claim that may contradict with what they believe in and consider that as part of the conspiracy itself, which is aligned with the fifth characteristic of CT. Overall, since the discussions in these climate change denialism communities demonstrate features of CT, we consider these communities as CT communities. To date, it has been challenging for researchers to interact with members of CT
communities because of skepticism toward scientists and institutions . For researchers to study the effects of educational and pro-science messages or debunking evidence in online conspiracy discussions such as climate change
denialism, it is important to explore communication strategies that allow researchers to maintain civil and mutually respectful conversations with CT community members. How can we introduce interventions that motivate conspiracy theory believers to explore scientific inconsistencies and fallacies in conspiracy theories? Building on the previous work observing the role of doubt and deliberation in disengagement from online CT discussion, this paper contributes a theoretically motivated intervention design framework that can be deployed in open discussions on social media. Specifically, we investigate how insider language could be used to contest socially constructed conspiracy theory ideology. Insider language refers to terminologies that people use to discuss conspiracy theories that are unique to the community For instance, the bolded phrases in Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway the examples above (e.g., “trace gas,” “sea level rise,” “temperature measurements,” etc.) are some specific terms people often use in these communities to describe climate change-related concepts. In this work, we ask:
- RQ1: How do CT believers (climate change deniers) respond to interventions contesting their beliefs?
- RQ2: How do climate change supporters respond in open discussions to interventions contesting climate change
denialism? Prior works in CSCW and the larger HCI community have long recognized the importance of bridging the gap between social requirements and technical capabilities . However, the functioning of CT communities from this
socio-technical standpoint remains understudied. Our work addresses this knowledge gap by examining both the social and technical infrastructures that shape participation in CT communities, specifically through the lens of members’
responses to interventions. We highlight the social infrastructure by analyzing how different stakeholders—climate change deniers and supporters—utilize evidence and claims in their debates, and how interventions could influence these interactions. This reveals the underlying mechanisms of community dynamics and identifies potential avenues for effective intervention engagement. Our work also points out the technical infrastructure, including the tools,
information sources, and policies that CT community members use in their interactions with other stakeholders and
interventions.
Often, conspiracy theorizing can lead to
real-world consequences such as inaction due to climate change denialism or harm to public immunity due to anti-vax
attitudes. In this paper, we focus on online discussions around climate change denialism and leverage the insider language in such online discussions to create interventions aimed at contesting the meaning-making process inside the climate change denialist communities. 2.1 Climate change denialism
Climate change, a scientifically established phenomenon, faces persistent challenges from conspiracy theories that aim to discredit its reality or downplay its significance. While the majority of the general public believe in global warming, there are still people who are skeptical or dismissive of the idea . For example, deniers may claim that temperature data is inaccurate or unreliable, suggesting that there is no significant warming trend
. Other climate change denial arguments reject the role of humans in climate change and instead point to natural
forces, such as the influence of the sun and cosmic rays . The climate change debate in the United States in particular is influenced by special interest groups and has affected discourse within the scientific community through contrarian narratives. More importantly, disbelief in climate change and the accompanying anti-science
attitudes have posed significant challenges for public policy design and corrective action. In this work, we examine climate change denialism from the lens of conspiracy theories and analyze key insider language terms that contribute to the meaning-making process of climate change conspiracy theories. 2.2 Conspiracy theories and insider language
Conspiracy theories often rely on the use of insider language or specialized terminology, contributing to the formation and reinforcement of group identity among believers. Conspiracy theorists may employ specialized language and coded symbols to communicate exclusivity among believers and establish boundaries between “insiders” and “outsiders”
. Online platforms contribute to the creation and dissemination of insider language within communities. Researchers have analyzed how insider language in conspiracy theories may evolve across platforms and contribute to internet-mediated radicalization. Given the important role played by insider language in conspiracy theorizing,
in this paper, we leverage the insider language of the climate change denialist community to craft intervention messages that challenge the concepts behind the established insider language. This process is inspired by the theory of social interventions described below. 2.3 Interventions for Conspiracy Theory Discussion
Several research studies have experimented with interventions aimed at addressing conspiracy beliefs. These interven
tions often focus on strategies to mitigate the impact of conspiracy theories and reduce their influence on individuals. For example, research has explored the effectiveness of debunking conspiracy theories by providing factual information and evidence-based refutations. However, studies have shown that mere exposure to corrective information may not always be sufficient. Other types of interventions also rely on inoculation theory. This approach involves
pre-emptively exposing individuals to weakened forms of conspiracy theories or misinformation, followed by debunking Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway
information. Studies applying this theory have found success in strengthening resistance against conspiracy theories by providing individuals with tools to identify and refute misleading claims. However, inoculation applied to specific misleading narratives, such as climate change denialism, may not be as effective for audiences that already support those narratives, known as therapeutic inoculation. The effectiveness of interventions can vary depending on individual differences, the nature of the conspiracy theory, and the underlying reasons for belief in conspiracies.
Moreover, the majority of the intervention experiments are conducted in controlled environments as surveys or user
studies. In this paper, we present one of the first works that deliver interventions in open, uncontrolled, and unstable social media spaces. Specifically, our interventions are inspired by the theory of social interventions which is described
below. 2.4 Social Interventions
The social intervention model from communication theory describes the dialogic social processes through which humans symbolically change the social systems. The social intervention theory refers to attention, need, and power mechanisms that bring out changes in ideologies and social processes. We specifically focus on the “attention”
mechanism that contests the socially constructed meanings to change ideologies.
For example, in their original work on the attention model, Brown describes how changes, or attention shifts, in the perception of Black identity in the early
1900s changed the racial relations in the United States. Conspiracy theories are filled with contradictions that, when realized by the believers, can lead to alack of belief and engagement in the conspiracy theory discussions. In this regard, drawing attention to the contradictions through dialogic social interventions may disrupt the conspiracy theorists’ meaning-making process and encourage conspiracy theory believers to critically reflect on their beliefs, potentially resulting in the goal of disengagement from the conspiracy community. In this paper, we craft interventions based on insider language of climate change denialism
in open public discussions and analyze the responses by the community members. 2.5 Countering Conspiracy Theories
While a systematic approach to counter conspiracy theories has not been studied before, some prior works did touch upon a population that is involved in this process – people who self-organize to counter misinformation and hate speech. For instance, Drolsbach et al. discussed how community fact-checking can help address
misinformation. Similarly, Twitter/X launched Birdwatch where communities of fact-checkers could collaborate and crowdsource to flag and address misinformation. Additionally, the Dangerous Speech Project exemplifies a resource where people volunteer to crowdsource resources against hate speech. These existing efforts demonstrate that there are many people who actively engage and try to address misinformation and conspiracy theories in the communities. When we deployed our interventions in the communities on Reddit, we also encountered a group of users who were similarly putting in their efforts. While they did not work together, they actively engaged with posts in the communities individually, providing rationale and reasoning for the pro-climate change perspective. We discuss them in more detail in section 4. Our work corroborates the existence of this group of users, and highlights future design opportunities to engage with this group in section 5.
3 METHOD
In the following section, we discuss how we designed, implemented, and deployed our interventions to selected
subreddits. We first used the SAGE method to analyze and identify insider language terms specific to the subreddits CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
Fig. 1. An overview of our intervention design and deployment process. The external events refer to events that occurred on Reddit that we could not control. we are studying. We then designed our interventions and iterated them through qualitative exercises to ensure their
effectiveness. Finally, we deployed our interventions in the subreddits in three phases and collected community members’
responses to our post. The entire steps are also shown in Figure 1. 3.1 Subreddits containing climate denialism
Our study focused on two subreddits: r/climateskeptics and r/conspiracy . r/climateskeptics was created in 2008 and has about 43k subscribers. The community focuses its discussions on questioning climate-related environmentalism and sharing pieces of news and evidence to refute climate change science. Since r/climateskeptics solely focuses their discussions on climate change conspiracy theories, this community was the main focus of our study. r/conspiracy was created in 2008 and has about 2 million subscribers. The community discusses a wide range of conspiracy theory-related
topics, including climate change and environmentalism. Prior research has studied these communities to identify conspiracy theory discussion dynamics and narratives. These communities were also selected because they explicitly permit bots in their community rules and have a community norm accepting bots that post in discussions. 3.2 Insider Language Detection
To ensure that our interventions were relevant and effective in r/climateskeptics, we first investigated how the community discussed the climate change denialism perspective. We first identified the “insider terms” of that community, which refers to the unique language used in the community. Prior research suggests that engaging people with these insider terms can be more effective in promoting critical thinking and challenging conspiracy theory logic.To
do so, we used the SAGE (Sparse Additive Generative Models) method, which has been adopted to extract and identify insider language in studying conspiracy theories .
SAGE is a method that uses a regularized log-odds ratio to contrast different text corpora’s word distributions, highlighting the words that are uniquely used in each text
corpora. In particular, we conducted a comparative analysis of r/climateskeptics against other subreddits where so-called ‘GreenHouse Gases (GHGs)’ – like CO2 – make it so that some of the energy that leaves the surface of the earth never manages to escape the system as a whole to space, but is rather ‘recycled’ internally between atmosphere and surface, creating ‘extra’ atmospherically induced warming of the surface on top of the original solar warming.”
Repeated comparisons to how mid-holocene climate was warmer than current mid holocene climate
Radiative cooling is possible for surface temperature, any warming from greenhouse gases is offset by more radiative cooling from Earth’s surface,
radiative cooling
“New paper finds greenhouse gases causing radiative cooling, not warming, at current Earth surface temperatures”
Table 1. This table contains examples of the insider terms we used. r/ conservative ) and pinpointed terminology distinctive to r/climateskeptics . We excluded words that were generic, that could have other meanings and those that were not frequently used in the last three months. We then conducted open coding on the identified insider language terms and highlighted how the community members share their insights and interact with one another. We then analyzed how these terms are used in anti-climate change context and brainstormed counter-arguments to these uses. 3.3 Intervention Design
Using the “insider terms” identified and the counterarguments brainstormed, we crafted a bank of intervention messages. We then conducted brainstorming sessions with 6 researchers to revise the messages so that the interventions would be more polite, and respectful. We then conducted a thematic analysis of the revised interventions. As common themes emerged, we developed sample responses used as input to ChatGPT to expand our set of interventions by generating additional similar comments, leveraging the language models to adapt the intervention messages to suit the specific context and tone desired. We used ChatGPT to generate responses because we intended to test the CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
scalability of LLMs in interventions.
All generated responses were still validated by humans afterward to ensure their quality and relevancy. For instance, ChatGPT generated an intervention such as the following:
There is a basic difference between mid-Holocene trends and current global trends. The mid-Holocene changes were much more gradual and over a 2000-year period, while current trends are at a much faster rate. This has been proven by researchers as well. https://www.rutgers.edu/news/important-climate-change-mystery-solved
scientists Do you think we should consider this faster rate of change to be a concern? In our human-validated edits, we added polite phrasings to make our interventions more approachable:
Thanks for sharing. You might consider whether there is a difference between mid-Holocene trends and current global trends. The mid-Holocene changes were much more gradual and over a 2000-year pe
riod, while current trends are at a much faster rate. There is some research on this you can check :
https://www.rutgers.edu/news/important-climate-change-mystery-solved-scientists What is your take?
Do you think we should consider this faster rate of change to be a concern? Additionally, to ensure consistency and minimize chances for reactance, we wanted to make sure that our interventions were consistently expressing a positive attitude. To control for this, we used EmoRoBERTa, a pre-trained Speaker-Aware
Emotion Recognition large language model, to identify the emotion of all the generated interventions. Specifically,
EmoRoBERTa is a fine-tuned transformer model that could detect the emotion in text. Given text input, it is able to classify whether the writer was confused, engaged, curious, neutral, and so on. We used EmoRoBERTa to calculate the emotional scores of the interventions . We then fine-tuned the interventions to make them demonstrate more positive emotions than negative ones.
Examples of the interventions generated are shown in Figure 2. Fig. 2. This figure contains four examples of the interventions deployed. Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway
3.4 Intervention Deployment
During this phase, we created 5 Reddit bot accounts to deploy the intervention, conducted pilot testing, and then deployed the interventions on two Subreddits. Below we outline the details of our deployment. 3.4.1 Bot Account & Karma Building. First, we created 5 Reddit accounts that we used to deploy the interventions. To automate the intervention delivery, we opted to use transparently labeled bot accounts, which allowed us to deliver multiple interventions automatically. To be considered as credible Redditors, we needed our accounts to meet age thresholds and had a sufficient amount of karma, which is “a reflection of how much one’s contributions mean to the community.” To curate sufficient karma so that the accounts could receive enough attention in the Reddit
communities, we conducted Karma farming. Karma farming is a common practice people use on Reddit to obtain karma points, where karma points reflects one’s authority and establishedness in the community. We adhered to Reddit’s
policy and made posts and comments regularly in communities about gaming, such as r/FUTMobile and r/HadesTheGame. Specifically, we provided constructive comments about the games and conversed frequently with community members in these communities. Over a few weeks, we curated about 100 karma points for each account, which is a reasonable level for us to demonstrate our credibility. 3.4.2 Deployment. We used the following approach to post our intervention messages to each subreddit community:
when an insider term was mentioned in a post, a bot account would reply with a corresponding intervention message regarding that term. We deployed the interventions in three phases. In the first phase, we tested if our deployment script worked accurately and if our deployment received responses from the community. Upon detecting comments that contained the insider language, our system would send email alerts about the posts. We would then review each instance and craft tailored responses to ensure that the intervention addressed the believers’ points. This pilot test lasted three days to ensure that our intervention deployed effectively. In the second phase, we deployed the interventions at a larger scale using two bots that automatically selected prompts and deployed them in the
community. To avoid posting repetitive or closely-timed comments from a single profile, we implemented a rotation system where the two automated accounts took turns performing the detection task at a 30-minute interval. In the final phase, we deployed our intervention at scale in r/climateskeptics . However, we encountered multiple setbacks,
including the Reddit blackout. During this process, we became too visible in r/climateskeptics so we chose to redeploy on another conspiracy community. Thus, we proceeded with our deployment in r/conspiracy . While our interventions were suitable for r/climateskeptics, they were not as effective in this community because r/conspiracy had slightly different insider language about climate change. We also encountered a few challenges with the Reddit platform throughout the deployment phase. We explain them in more detail in Appendix A as a reference for future work. Despite the challenges,
we completed our deployment on Oct 6th 2023. After completing the deployment, we proceeded with data analyses to understand the challenges and opportunities in intervening in conspiracy theory communities. 3.5 Data Collection & Response Analyses
3.5.1 Data. After the final deployment, we collected every response, including a total of 44 follow-up discussions in the thread. In total, we deployed 62 interventions on Reddit. 42 of these interventions received responses. 29 of the responses were by the original poster. Figure 3 shows an example intervention and people’s response to it. Additionally, we observed that 36 of 46 interventions deployed in r/climateskeptics were relevant. Some of our interventions misfired because the participants had been discussing CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
different topics than the post that invoked the insider term. Figure 4 shows an example of an out-of-context example. While the original poster mentioned the term “chaos theory,” the discussions in the post was not about this in the context of climate change. Among the 16 interventions deployed in r/conspiracy, 6 were relevant. As we mentioned in the previous section, this phenomenon was more prominent with our interventions deployed in r/conspiracy because we built our interventions using insider language from r/climateskeptics . These terms were slightly different from those of r/conspiracy . We removed the out-of-context interventions from data analysis.
In total, we had 42 data points for
analyses. Fig. 3. An example of an intervention deployed in r/conspiracy that directly addressed the original poster’s argument. 3.5.2 Analyses. First, we analyzed all the original posts that we intervened on and identified two groups of Redditors
that we interacted with. The first group are climate change deniers, who rejected scientific consensus on climate change. The other group are climate change supporters, who supported our post.. We describe the characteristics of each population in more detail in subsection 4.1. Considering the original posters and all the
Redditors who engaged in follow-up discussions, we interacted with 72 Redditors. Among these Redditors, 57 were climate change deniers, and 15 were climate change supporters. Among the 42 relevant interventions deployed, 30 were deployed to climate change deniers and 12 were deployed to pro-climate change users. We conducted a thematic analysis of posts by each group separately. The authors developed a codebook by analyzing extracted excerpts from the responses to our interventions. Throughout the analysis process, one researcher first performed individual analysis and extracted excerpts. Then three researchers discussed the excerpts extracted until they decided on a final codebook. The final codebook includes themes such as climate change deniers’ response to evidence
based intervention, climate change deniers’ response strategies, and climate change supporters’ response strategies.
One researcher then used the codebook to code all the data collected. Throughout the coding process, the researchers analyzed Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway
Fig. 4. An example of an intervention in r/consipracy that was out of context because the keywords detected were not associated with the climate change context. climate change deniers’ and supporters’ posts separately. In the analysis of deniers’ posts, we focused on understanding how the deniers responded to the interventions. Additionally, to understand any variation in emotional reactions that original posters and responders had to our intervention, we analyzed the collected data using EmoBERT . Recall that
EmoRoBERTa is a fine-tuned transformer model that could detect the emotion in text, classifying the writers’ emotion when writing the text. Given text input, EmoRoBERTa provides granular emotion labels such as confused, engaged,
curious, neutral and so on. To make it clearer for the analysis, we classified these labels into three categories: positive,
neutral, and negative. For instance, engaged is considered as positive, curious is considered as neutral, whereas anger is considered as negative. Specifically, we contrasted the emotion between the original post and the response to our intervention to understand if our intervention had any impact on users’ emotions in the discussion. We included the results of believers’ posts in Figure 7. In the analysis of the supporters’ posts, we focused on how they interacted with our interventions, and how our interventions helped them reinforce their opinions in the community. We conducted the same EmoBERT analysis with pro-climate change Redditors’ posts. However, we did not observe significant correlation as we did with believers’ posts. 4 RESULTS
4.1 Characterizing user population who engaged with our intervention
Our data analysis revealed that there were two groups of users who interacted with our intervention— climate change deniers and climate change supporters . In the following section, we discuss the characteristics of each group, including how they interacted with our post. 4.1.1 Interventions to Climate Change Deniers. We define climate change deniers as those who oppose the opinions of our intervention and believe in anti-climate change. In particular, deniers expressed strong opinions about climate change scientists and the evidence they provided. For instance, D26 stated that “there is a claim ice is melting” but that CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
Theme Definition
Deniers: evidence Intervention Contained Evidence:
Intervention: “I value your perspective. However, have you thought about the current rapid escalation in sea levels? While the mid-holocene saw fluctuations, the pace of change was much slower. Shouldn’t this be a crucial factor in the climate change debate? See here [Linked Resource]”
Response: “I’m sorry my friend I do not believe that sea levels are different ...stuff still looks just the same as it did when I was a kid.”
Intervention Contained No Evidence:
Intervention: “Do you believe that alarmist language around climate change could be a result of the growing body of evidence? Is it possible that these new terms help us better understand the issue?”
Response: “There is no body of evidence presented here so the ‘new terms’ only serve to obfuscate the fact.”
Deniers: original poster’s emotion
Deniers: responses to intervention with no mention of
terminologies
Supporters: respond ing with more ratio nale
Supporters: Re sponding with more pro-climate change evidence
Original poster was emotional:
Intervention: “Your concerns about alarmist narratives are valid, but have you delved into the compelling information offered by climate science? Might this not provide us with a more factual perspective on our changing climate, apart from alarmist views? See here [Linked Resource]”
Response: “Evidence of climate change? ...Compelling bullshit is what this is.”
Original poster was calm:
Intervention: “That’s an interesting perspective. Do you think we should be considering the rate of sea level rise, in addition to the absolute change, when evaluating the impact of climate change? While sea levels during the mid-holocene were different, the rate of change was much more gradual than what we’re experiencing today. See here [Linked Resource]”
Response: “The arctic ice melt is further evidence of this. The rate of change is still very slow today. Certainly it’s slow enough to adapt to it calmly.”
“Alarmist” was mentioned:
Intervention: “Your caution against alarmist dialogue is noted, but have you explored the breadth of data that climate science offers? Could this not expand our understanding of climate change, leaving alarmist exaggerations aside?”
Response: “Alarmist dialogue is wrong. Have you heard of a thing called the fossil record? In which extremes in climate are forever recorded? Let’s include that in any discussion on climate change.”
Intervention: “It’s fascinating to explore the scientific evidence linking greenhouse gases and the Earth’s temperature, especially in the context of climate change. What’s your take on the topic?”
Response: “Infarered energy escapes through water vapor. Water vapor isn’t the issue. It’s co2 and the methane and other harmful gases that us humans have unnaturally put into the atmosphere.”
Intervention: “Your input is important! How does the scientific evidence supporting the role of greenhouse gases in the Earth’s temperature impact our discussions about climate change and its potential causes? See this link [Linked resource]”
Response: “Understanding greenhouse gases is essential to understanding the current climate crisis. [Linked resource]. Accelerated climate change is threatening the way we go about life ...Businesses will have to adapt quickly.” 12
Table 2.
They do, by the way, deliberately lie and falsify data, it’s very well documented. So forgive me for being skeptical about a group of proven liars and con-artists.” In contrast, if our intervention just responded to the topic discussed instead of promoting the conversation to involve terminologies, the deniers showed a more positive attitude. When our intervention simply expressed a concern building on the original poster’s argument and said “How do you think the faster rate of sea level rise today compares to the rate seen during the mid-Holocene, and what does this mean for discussions about climate change,” deniers did not immediately shut Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway
Fig. 7. This image shows the EmoBERT analysis result of the comment (original post) and the response to the corresponding intervention. For the original post and its corresponding responses, we grouped the analysis result into three categories: positive (e.g., optimism, curiosity, etc.), neutral, and negative (e.g., anger, annoyance, etc.). We considered the negative posts as a reflection of the deniers showing a strong emotional investment in the conspiracy theories. In contrast, if the posts expressed a neutral or positive attitude, we considered that the deniers expressed a low emotional investment. Note that this graph only showcases the analysis of climate change deniers’ original posts. down to discussion. Instead, D26 responded to our question by redefining “the claim ice is melting” as the basis of the problem. B26 then continued to introduce the rationale that “there are a huge amount of papers declaring what should happen yet it has not” to showcase how the pro-climate change perspective may fail here. In extension, D26
brought up an example of how “the Fremantle harbor was developed from a shallow delta with a channel that the first explorers sailed up and found the Swan estuary which had freshwater springs and a sand bank and became Perth.” In this exchange with our intervention, the deniers showed a more open attitude of sharing their own thoughts and their own set of evidence of why they believed in the conspiracy theories instead of fixating on the specific terminologies and elevating the discussion to an emotional level. 4.2.3 Responding depending on original poster’s emotion. Using EmoBERT to test our data’s emotional level,
we found that if deniers expressed a neutral and/or curious attitude towards climate change denialism instead of deeply invested in them, they would be more likely to engage in a back-and-forth knowledge exchange with our bot
account. And after the interaction with our intervention, 11 out of the 15 deniers (who showed a neutral and/or curious
attitude) maintained their neutral and/or curious attitude, demonstrating that our interventions kept them open to conversations and further engagement. Additionally, 4 out of the 15 deniers expressed a more positive attitude after the interaction, showing a possible increase in how open they are to evidence and further conversations. For instance,
when our intervention showed the “current rate of sea level rise” with linked evidence, D26 who was asking questions about how to think about the “polar bears which were meant to be in decline and they are not,” and asked “how do I
think about this problem” responded to our evidence with a well-defined discussion.
Similarly, in an exchange with
D2, we found that when the deniers held a more curious attitude, they are more likely to engage in an intellectual
conversation. When D2 cited evidence explaining no increase in natural disasters causally lead to no climate change, CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
D2 used the phrase “Can you show me where he is wrong?” When we provided linked evidence, D2 responded by engaging with the citation, explaining how our evidence failed to address the issue: “Yearly changes in sea level are below the measurement accuracy of the instruments. This citation is saying ice is melting at the fastest rate and yet no one can point to any place on earth.” Though D2 was still entrenched in climate change denialism, D2 was willing to engage in a discourse with our post. Based on Figure 7, in most cases, deniers’ emotional state improved or remained the same. However, one case came to our attention where the denier’s emotion changed from positive to negative after encountering our intervention. Initially, D40 was sharing a detailed explanation about how the rise in population caused the temperature rise instead of climate change. In our intervention, we mentioned that “There is a consensus among scientists that greenhouse gases play a significant role in climate change.” Then D40 responded with “There is no consensus, only bought-off scientists.”
This reinforced our result discussed in subsubsection 4.2.2 that when intervention contained certain terminologies,
deniers may be triggered and respond emotionally. 4.3 RQ2: How do climate change supporters respond to our interventions? Our analyses did not show any direct correspondence between the original post, our intervention, and the corresponding responses (e.g., whether the intervention contained evidence, whether the original poster was emotional, or whether the intervention contained certain terminologies). But we did observe that climate change supporters generally showed a positive attitude toward our intervention. Our intervention allowed them to more easily discuss their pro-climate change perspectives and offered them the opportunity to provide more evidence. Examples of climate change supporters’
responses are shown in Table 2. 4.3.1 Responding with more rationale. Climate change supporters in the community used our intervention as away to extend the discussions of the pro-climate change perspective. When S12 was arguing for the green movement, our intervention introduced the topic of greenhouse gases’ role in the Earth’s temperature and climate. S12 then asked the following question: “Tell me what you know about greenhouse gases.” The response demonstrated S12’s willingness to share more thoughts on the given topic. Furthermore, when we introduced pro-climate change evidence, S2 built on our intervention and discussed how the article explained the greenhouse effect. Using phrases such as “that passage provides a great historical account of,” S2 shared additional insights about how new evidence such as the article “reflects
the evolving nature of scientific inquiry and the refinement of theories.” Similarly, when we introduced evidence about greenhouse gases and a climate change denier in the community responded with “There’s no consensus.
Only among bought-off scientists.” S5, a pro-climate change Redditor, stepped in and argued in our favor: “No there’s a
consensus. Whether the consensus is 97% of climate scientists or 88% that’s still a very strong consensus.” In this
interaction, our intervention received both climate change deniers’ and supporters’ attention, which encouraged more discussions and debates in the community. 4.3.2 Responding with more pro-climate change evidence. Our intervention also allowed climate change supporters to offer more evidence in follow-up discussions. When our response highlighted “alarmist messages” in response to a
pro-climate change argument around temperature change, S8 responded with “The situation is alarming. We should all be alarmed” followed by a list of detailed examples of temperature change, such as “Canada’s wildfire as of today have already burned more land than in any previous wildfire season,” and “Texas was under a 45-degree Celsius heat dome for two weeks, killing dozens of people.” In this interaction, the climate change supporters built on our intervention to reinforce their own argument.
Similarly, when our intervention corroborated S10’s post on sea levels are rising Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway by asking the question of “Can we discuss the current trend of sea level rise and its implications for climate change? The rate of change today is much faster ...indicating the impact of climate change.” S10 showed a positive attitude and shared more evidence supporting how “ice levels are connected.” Further, as cited sources are provided, N10 also followed up with descriptions and rationales: “the ice caps reflect some sunlight in the visible range. As there is less ice,
less gets reflected and so total warming increases.” Overall, our engagement allowed the climate change supporters to provide more evidence and reinforce their argument in the climate change debate. 5 DISCUSSION
In this work, we studied how to intervene in conspiracy theory communities and how these interventions may influence the way community members engage in climate change debate. Our work took a novel approach to design and deploy the interventions. By interacting with two groups of users, climate change deniers and climate change supporters, we identified what types of interventions worked well in these communities. We also gained insights about how climate change supporters engage in these conspiracy theory communities, pointing out how future work could study and engage with this population. 5.1 Designing Conspiracy Theory Interventions
5.1.1 Community-centric Interventions. In online communities such as Reddit, the community as a whole shapes the way that people speak about the conspiracy theory and how they interact with one another around it. Many existing platforms such as Twitter/X has focused on engaging with the spread of misinformation from an individualistic perspective,
where each account and each post is addressed as misinformation. Furthermore, many existing works studying the design of social interventions focus on targeting individual users as well. Our work instead took an approach to engage the community as a whole because the social context (where people encounter conspiracy theories and how others engage with them in that context) is an important aspect contributing to intervention design. We demonstrated the possibility and importance of thinking about groups of conspiracy theorists as communities. We tried to design our interventions as closely related to the community as possible by engaging with insider language unique to that community. We also used GPT-4 to generate variations of the interventions so that the community members would not feel like they were engaging with an automated account. Additionally, we attempted to simulate the intervention experience as close to interacting with areal member of that community as possible because of the motivated reasoning phenomenon. Prior works suggest that the motivated reasoning phenomenon may occur if the conspiracy theorists experience an identity threat based on the provided evidence . In that case,
conspiracy theory believers are more likely to engage their defense mechanism and choose to continue holding their original belief. Our approach engaging with insider language helped us better align with believers’ identity and thus avoid the motivated reasoning effect. Our focus on insider language not only enabled us to meet the community where they were, but also helped us understand the discussion themes in these climate conspiracy communities. Overall, our results indicate the benefits of using community-centric interventions to engage with conspiracy theories. However,
since our approach used a manual approach to collect and counter the insider terms, future work could explore how to automatically detect and analyze these insider languages unique to each community. This could make the process of designing interventions easier and more efficient. 5.1.2 Evidence-based Interventions. Providing evidence to support the pro-climate change perspectives was an important component of our intervention. Deniers showed a higher tendency to respond and engage when evidence was provided. CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
However, existing literature pointed out that these communities may be prone to having scientific argumentation,
engaging in evidence provided by others through certain rationales to support their own claims. Future work could deploy interventions in a wide range of conspiracy theory communities to further understand the generalizability of the approach to use evidence. Additionally, when designing our interventions, we carefully considered how to best include the evidence since prior works suggested that the backfire effect could impact the conspiracy theory believers’ consumption of information. As existing works noted, the backfire effect occurs when people fail to consume information objectively, which means they are less likely to change their opinions about the original information. Thus, we fine-tuned our interventions to be as respectful and polite as possible when raising opposing opinions and evidence. Future works intervening in these communities should also account for the possible impact of backfire effect when introducing evidence. 5.2 Intervening climate change deniers
In our study, we deployed our interventions in subreddits and observed how users interacted with our posts.
After analyzing the collected data, we observed that there are two groups of Redditors in the community: climate change deniers and supporters. Our results indicated that when interacting with climate change deniers, the most important component of an intervention is linked evidence. When climate change deniers engaged with our interventions, they directly engaged with the provided evidence. In fact, deniers often explicitly ask for evidence if someone is proposing an opposing view. Without the evidence in the intervention, more back-and-forth discussions about what the evidence is and where the evidence is from would ensue. Our results echo prior research in other domains (e.g., health, psychology, education,
etc.), showcasing that evidence is a key component in communicating ideas with others, especially if the goal is to intervene others’ perspectives and the evidence is introduced in a respectful way. When deploying the interventions, another important aspect is to avoid certain terms such as “climate change
scientists.” Explicitly mentioning the pro-climate change perspectives could lead to a complete shutdown towards any evidence and rationale posted in an intervention. Additionally, we found that how emotionally invested the original poster was, influenced how the poster responded. As explained in subsubsection 4.2.3, when the emotional investment was higher, there was a higher likelihood that the person would respond negatively. We also observed that climate change deniers were willing to share their personal stories to corroborate their point of view. Prior research has shown that personal stories can invoke emotional resonance with the audience. Leveraging existing discussions about how to share a personal story and highlight human values in response to conspiracy theories, future research could explore how to best use an automated system to generate such interventions. Additional research could also investigate how to increase people’s exposure to alternative points of view. For instance, an automatic system could be developed to respond to conspiracy theorists even after they started reacting emotionally. This approach could collect more insights about how conspiracy theorists engage in an emotional state, and help explore how to best engage with them in these situations. Moreover, our work indicated that whether the conspiracy theory believers are emotionally invested is an indicator of how they would respond to interventions. Prior works have also suggested that “emotional
selection” influences how people consume content .
Specifically, conspiracy theory believers are more likely to engage with content that correspond to their emotional state. Similarly, Heath et al. suggested that emotion snowballing could occur when intense emotion is spread from one another . To more closely investigate how emotion influences people’s perceptions of the interventions, future work could conduct experiments to compare how people respond across situations of emotional selection, emotion snowballing and so on. Another aspect worth exploring is how to Towards Designing Social Interventions for Online Climate Change Denialism Discussions CSCW, October 18–22, Bergen, Norway sway the “movable middle.” The “movable middle” are people who have not decided about whether they would believe in the conspiracy theories yet. Future work could study this population specifically and design the interventions more tailored to their perspectives. 5.3 Interacting with climate change supporters
Prior studies have identified the existence of a group of users who actively counter misinformation and hate speech
online. Our work highlighted a similar group of users who actively engage with conspiracy theories. Specifically, our work revealed how pro-climate change Redditors interacted with members of conspiracy theory communities and how our interventions supported their efforts by providing additional evidence for climate change. Our work underscored that people who do not hold the belief of the community may play an important role in encouraging more open discussions in the community. Future work can more specifically focus on counter-attitudinal
populations, or pro-climate science populations, and explore how to best support them in engaging effectively in conversations in conspiracy theory communities. They could explore the specific needs of these users, including how to support their process of collecting information, how to support their process of organizing that information, and how to support their process of interacting with conspiracy theory believers.
For instance, based on Twitter/X’s Birdwatch
project and the Dangerous Speech Project, there is an interesting opportunity to explore how to aggregate a central hub of information countering conspiracy theories to help alleviate these users’ burdens. 6 ETHICAL CONSIDERATIONS
Throughout the design and deployment of the study, we tried to minimize harm. Specifically, we were interested in engaging with conspiracy theory community members to explore whether and how to change their beliefs. To do so,
we needed to build trust with the community members. However, this is difficult to do because these communities do not trust scientists at all, which is one reason why they believe in conspiracy theories. Thus, instead of running a full deception study, we chose a middle ground. Building on existing research deploying interventions to misinformation and conspiracy theories, when we created the accounts for deployment, we made sure to self-disclose
the accounts as bots used for research purposes instead of deceiving the community members. However, since we did not explicitly disclose our accounts as bots when we posted interventions each time (the information that we were researchers was only listed under the account profile), some community members may not have realized that we were conducting research. We believe that future research should also ensure transparency using approaches such as account labeling when deploying interventions. When we collected data from the Reddit communities and used the SAGE
method to identify insider language, we made sure all data was anonymized. When we analyzed the collected data
(responses to our interventions), we ensured that all users were anonymized. We also made sure that we only collected and analyzed users’ posts in the community and did not collect any de-identifiable information from them. Second,
when we designed our interventions, we also tried to fine-tune them to be polite and respectful. 7 LIMITATIONS
There are some limitations to this work. First, our sample size was relatively small due to several external challenges we faced in the deployment phase, such as the Reddit blackout. Additionally, while lots of our interventions received
upvotes and downvotes from community members, some did not receive a response in the text format. Thus, we had a relatively small sample to perform qualitative analysis on. Future large-scale deployment could help resolve these issues and provide more quantitative insights. Additionally, our work only explored how people responded to our intervention. CSCW, October 18–22, Bergen, Norway Ruican Zhong, Shruti Phadke, Beth Goldberg, and Tanushree Mitra
We did not further engage with people who responded with follow-up conversations. Future work should study these
follow-up interactions more closely to further understand how to best engage with conspiracy theory believers. 8 CONCLUSION
In this work, we designed and deployed interventions addressing conspiracy theories in Reddit communities. We designed interventions tailored to the conspiracy communities by addressing the specific insider terms they used. Our work highlights the process and challenges of automatically intervening in conspiracy theory communities. Additionally, throughout the deployment, we encountered a group of users who voluntarily engaged with conspiracy
theories, highlighting future opportunities to study and engage with them. Overall, our work provides implications for future works studying and interacting with conspiracy theorists. 9 ACKNOWLEDGEMENT
Research reported in this publication was supported by Google Jigsaw. We would also like to thank Neelesh Azad for helping us with the experimental design that supported the methodology write-up of the paper.