Forecasting accuracy and confidence in the context of the COVID-19 pandemic
Creators: name: Mason, Liam
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Social and Behavioral Sciences

description: Author(s): Liam Mason – l.mason@ucl.ac.uk Steve Fleming – stephen.fleming@ucl.ac.uk Alex Lloyd – Alex.Lloyd.2018@live.rhul.ac.uk Ryan McKay – Ryan.McKay@rhul.ac.uk 1) Data collection. Have any data been collected for this study already? Data collection is in progress; no data will be analysed until collect completes on 11/05/20. 2) Hypothesis. What's the main question being asked or hypothesis being tested in this study? Our overarching research aim is to study information processing and decision-making in the context of the COVID-19 pandemic. Our specific predictions here concern forecasting accuracy and associated confidence, and how this drives real-life health-related behaviours, particularly adherence to social distancing. Our assumption is that people are on average poorer at predicting rapidly changing environments; in this case environments in which growth is non-linear. In addition, people are likely to be over-confident about their predictions in these environments, particularly early on in learning. We therefore have three key hypotheses each with specific predictions: 1. Basic task effects (H1): H1a) Participants will show lower accuracy in their predictions in the non-linear environments, in line with a previous behavioural economics study (Arnott &amp; O’Donnell, 2008). H1b) Participants will be more confident in their predictions in the non-linear environments compared to linear. This over-confidence will be especially evident in early predictions (first third of trials). 2. Confidence about COVID-19 pandemic (H2): H2a) Participants who are more accurate in their predictions about changing environments will be those reporting lower confidence about being able to predict how things will progress with the COVID-19 crisis (near transfer item). This will be evident in both environments: linear and exponential, but especially in the case of exponential growth which most clearly approximates the current pandemic in terms of number of cases/deaths arising and secondary effects (tightening of social distancing measures and economic implications). H2b) Higher levels of early over-confidence about one’s own predictions will predict 1) greater confidence in an ability to predict how things will progress with the COVID-19 crisis (near transfer item) and 2) reduced adherence to social distancing. 3. Mood symptoms (H3): Higher levels of early over-confidence about one’s own predictions will predict lower levels of worry about COVID-19 and lower levels of generalised anxiety (GAD-7). 3) Dependent variable. Describe the key dependent variable(s) specifying how they will be measured. Forecasting accuracy and confidence will be assessed using a task adapted from Arnott &amp; O'Donnell (2008). Participants are periodically asked to predict the next number in a sequence, and to indicate their confidence. This task will be administered as part of a wider online task battery. We will administer the following self-report measures: • Trait Catastrophising Questionnaire (short form; Pike et al, preprint) • Generalized Anxiety Disorder questionnaire (GAD-7; Spitzer et al., 2006) • Patient Health Questionnaire depression scale (PHQ-8; Kroenke et al., 2009) • Mood Zoom, a momentary assessment of mood states (MZ; Tsanas et al., 2016) • Hypomanic Personality Scale (HPS; Eckblad &amp; Chapman, 1986) • A series of bespoke items tapping 1) worry about the Covid-19 pandemic and 2) social distancing and other health behaviours (e.g. handwashing) in relation to COVID-19 with ratings given for “now” vs “usually (i.e. prior to the pandemic)” to derive change scores. 4) Conditions. How many and which conditions will participants be assigned to? They perform two repetitions of three conditions (order counterbalanced) in which the growth is 1) linear, 2) logarithmic or 3) exponential. 5) Analyses. Specify exactly which analyses you will conduct to examine the main question/hypothesis. 1. Basic task effects (H1): To ascertain whether accuracy (H1a) and confidence (H1b) differ by condition, we will test for a main effect of condition and condition x rating interaction using repeated measures ANOVA. This will be performed separately for accuracy and confidence, with factors: growth condition (linear, logarithmic, exponential), repetition (first, second), and rating number. 2. Confidence about COVID-19 pandemic (H2) and mood symptoms (H3): Regression analyses with predictor variables 1) forecasting accuracy and 2) confidence. The predicted variables will be H2) self-reported confidence about how the pandemic will progress and adherence to social distancing (change from usual) and H3) worry about COVID-19 pandemic and generalised anxiety. 6) Outliers and Exclusions. Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. N/A. 7) Sample Size. How many observations will be collected or what will determine sample size? We will recruit a representative subset (target n=400) of the second wave cohort of the COVID-19 Psychological Research Consortium (C19PRC) Study. C19PRC is a longitudinal, multi-country study that aims to assess the psychological impact of the COVID-19 virus in the adult general population, commencing in the UK (McBride et al. 2020). The first wave of the study, C19PRC-UKW1, involved the recruitment of a large representative sample of UK adults (N=2025; quota sampling methods ensured that the sample was representative in terms of age, sex and gross household income), and was conducted by the survey company Qualtrics between 23-28 March 2020. The second wave of the study, C19PRC-UKW2, was launched 22 April 2020. Qualtrics will contact a subset of the C19PRC-UKW2 cohort and will embed our tasks which will be hosted on a partner platform (Gorilla). 8) Other. Anything else you would like to pre-register? N/A. 9) Name. Give a title for this AsPredicted pre-registration Forecasting accuracy and confidence in the context of the COVID-19 pandemic References Arnott, D., &amp; O'Donnell, P. (2008). A note on an experimental study of DSS and forecasting exponential growth. Decision Support Systems, 45(1), 180-186. Pike, A. C., Serfaty, J., &amp; Robinson, O. J. (2020, March 24). The development and psychometric properties of a self-report Catastrophising Questionnaire. https://doi.org/10.31234/osf.io/h726k