Belief change in response to evidence: Predictors and persistence over time (Study 2)
Creators: name: Anglin, Stephanie
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Social and Behavioral Sciences

description: Research Questions: Does belief change in response to empirical evidence persist over time? What variables are associated with belief change in response to empirical evidence? Study Overview/Design: This study is a conceptual replication of Study 1 on a different stimulus study topic (gun control). In Study 2, participants will read about a real study examining the relationship between state gun control laws and murder rates. Participants will be randomly assigned to read evidence suggesting that gun control is associated with lower or higher murder rates. Participants will only read one stimulus study in this experiment. Beliefs will be measured before and after the presentation of the evidence, along with at a follow-up 24 hours later. Possible individual difference variables associated with belief change in response to empirical evidence will be measured. In this design, the primary factors of interest are the stimulus study findings (gun control associated with lower or higher murder rates) and time of belief assessment (T1, T2, T3). Procedure: Participants will be recruited from Amazon’s Mechanical Turk for a two-part study. Session 1. Participants will first report their position on gun control (1=strongly oppose, 7=strongly support; position T1), belief about how background checks on gun sales are associated with gun-related murder rates (1=much lower, 7=much higher; correlational belief T1), belief about how background checks on gun sales affect gun-related murder rates (1=strongly decrease, 7=strongly increase; causal belief T1), and the certainty of their (causal) belief (1=very uncertain, 7=very certain; certainty T1). Participants will also answer a few distractor questions and complete a scale assessing their current emotions. Next, participants will be presented with a summary of a recent study examining the relationship between state gun control policies and gun-related murder rates. Participants will be randomly assigned to one of two conditions, in which the study suggests that gun control is associated with lower or higher murder rates. In both conditions, the evidence presented is summarized from a real study, so there is no deception. Following the study summary, participants will evaluate the study in a closed (2 items, 7-point scales; “How well was the study conducted?” and “How convincing was this study?”, which will be averaged) and open-ended manner (i.e., “Why do you think the study you just read does or does not support the argument that gun control [decreases/increases] gun-related murder rates?”). Then, participants will again report their emotions, position on gun control (position T2), belief about how background checks on gun sales are associated with gun-related murder rates (correlational belief T2), belief about how background checks on gun sales affect gun-related murder rates (causal belief T2), and the certainty of their (causal) belief (certainty T2). They will also be asked to recall the study findings (included as an attention check). In the second part of Session 1, participants will complete several individual difference measures: social desirability (Crowne &amp; Marlowe, 1960), actively open-minded thinking (Haran, Ritov, &amp; Mellers 2013), perceptions of scientific (un)certainty, a measure of commitment to one’s beliefs about gun control, and scientific reasoning ability (Drummond &amp; Fischhoff, 2017). These measures will be presented in randomized order, with the exception of the scientific reasoning scale, which will be presented last (due to the difference in its formatting). Because two factors emerged from the scientific (un)certainty scale developed in Study 1, but only 2 items loaded on the second factor (and thus had low reliability), an additional 4 items were added to further assess this component of the scale. At the end of Session 1, participants will answer several demographic questions. Participants will be paid $2 for completing Session 1. Session 2. Approximately 24 hours after completing the first session, participants will be sent a link to complete a brief additional survey. In this survey, they will again be asked to report their position on gun control, beliefs about how background checks on gun sales are associated with and affect gun-related murder rates, and the certainty of their (causal) belief. Participants will be paid an additional $1 for completing Session 2. Copies of the Qualtrics surveys for each session (with exact question wording) are available on OSF. Power Analysis: For a 2 (research findings: gun control associated with lower vs. higher gun-related murder rates) x 3 (time: T1, T2, T3) mixed-model ANOVA, a sample size of N=116 is necessary to obtain small-medium effects (f=0.15) at power of .95, alpha=.05. The power was set high (.95) to ensure the quality of the replication (Funder et al., 2014). Because it was anticipated that many participants would fail to return to complete Session 2 of the study (administered 24 hours after Session 1), an initial sample of N=300 was targeted for Session 1. Planned Analyses: Exclusions. The main analyses will be performed with and without those who fail the attention checks and provide nonsensical/off-topic responses to the open-ended questions. Any discrepancies in the findings based on whether participants are excluded will be reported. If no discrepancies exist, all participants will be retained in the results reported. Belief, Position, and Certainty Change. 2x3 mixed-model ANOVAs, with research findings as a between-subjects factor and time of belief/position assessment as a within-subjects factor, will be performed to examine whether participants change their beliefs (or position) in response to each study and maintain this shift 24 hours after the presentation of evidence. Although the primary test of the hypothesis is the analysis for beliefs, position will also be examined as it has shown change in previous studies, though to a lesser degree (Anglin, 2019). Planned contrasts will be performed comparing T1 to each subsequent time point (T1 vs. T2, T2 vs. T3). To control for multiple comparisons, the significance level for all planned contrasts will be set to p &lt; .01. Hypotheses. Based on the results of Anglin (2019) and Study 1, an interaction between research findings and time of assessment is expected, with participants shifting their beliefs (and position) in response to the evidence presented and maintaining this shift 24 hours later. Note: The evidence presented is correlational rather than causal, and some participants may not shift their beliefs about whether gun control decreases or increases gun-related murder in response to the evidence for this reason. Therefore, in Study 2, an additional belief question was added at each time point to assess participants’ (correlational) belief about the association between gun control and gun-related murder rates. This question will be examined in an exploratory fashion, as it is unknown how strongly correlated responses to the correlational and causal belief questions will be. Belief and position change based on the (in)consistency of the evidence with prior views. Beliefs at T1 will be recoded so that higher scores indicate a stronger belief in the direction of the evidence presented, and belief change scores will be calculated (from T1 to T2 and T1 to T3) so that higher scores indicate more belief change in the direction of the evidence. Then, regression analyses will be performed, with the recoded belief at T1, research findings, and their interaction as predictors of belief change, to examine whether participants exhibit more or less belief and position change based on the consistency of the evidence with their initial belief (or position). Hypotheses. Based on the findings of Study 1, participants are expected to show the most belief and position change when the evidence conflicts with their prior views (presumably because these individuals have more room with which to move). Study evaluation ratings. A regression analysis will be performed, with belief at T1, research findings, and their interaction as predictors, to examine whether participants evaluate the evidence more favorably when it is more consistent (vs. inconsistent) with their beliefs. Hypotheses. An interaction between belief at T1 and research findings is expected, such that participants are expected to evaluate each study more favorably when it is more consistent with their initial belief. Open-ended evaluations. Participants’ open-ended evaluations will be examined in an exploratory fashion. Correlates of belief change. If the items from each individual difference measure have good internal consistency (i.e., α ≥ .70) and all items load ≥ .40 on a single factor, they will be averaged to create a composite measures. If not, items with low item-total correlations will be dropped and/or multiple factors (as determined by a factor analysis) will be examined separately. Correlates of belief change in response to the evidence (from T1 to T2 and T1 to T3) will be examined. If the two belief items correlate strongly (i.e., r &gt; .70), they will be averaged; otherwise, they will be analyzed separately. Correlations will be conducted between each belief change and individual difference measure (study evaluations, social desirability, positive and negative emotion at T1 and T2, actively open-minded thinking, scientific reasoning, belief commitment, and perceptions of scientific uncertainty). Exploratory analyses will also be performed to examine whether belief (in)consistency moderates these relationships. Hypotheses. Based on the findings from Study 1, study evaluations are predicted to correlate positively and perceptions of scientific uncertainty negatively with belief change for the causal item.