Statistical Inference Making
Creators: name: Seabright, Paul
name: Goksel, Selin
Publisher: name: OSF Registries
Year: 2020
Subjects: subject: Business
subject: Marketing
subject: FOS: Economics and business
subject: Economics
subject: Social and Behavioral Sciences
subject: Psychology
subject: FOS: Psychology

description: We explore people’s ability to make statistical inferences and how this influences their future decisions in stochastic environments. In particular, we are interested in understanding the factors that influence people’s ability in individual and social learning contexts, and comparing how their performance is modified by the possibility of learning from others. In both contexts, we explore the influence on the quality of statistical inference of (a) traits of individual participants and (b) characteristics of the decision task.
The first experiment in our program tests the effect of these factors on subjects’ ability to make statistical inferences in a neutral setting without the possibility of social learning. In order to do this, we will present participants with an urn containing 100 balls in two different colors (blue and yellow). Participants will then make 12 consecutive draws from this urn with replacement, and will be asked to make 3 consecutive guesses about the true proportion of the balls in the urn (i.e. they will make a guess after every 4 draws). We will manipulate the true proportion of the balls by randomly selecting a number from a uniform distribution between 0 to 100 for each participant at the beginning of the experiment. We will also ask randomly about the proportions of blue balls and of yellow balls so as to avoid anchoring based on one color’s being systematically the subject of inquiry.
After the participants complete this statistical task, they will be asked to report three different sets of individual characteristics as independent variables. The first set measures personal characteristics of an individual, which includes a scale for the Big Five personality traits, a single item measure of competitiveness, a single item measure of risk aversion, and a 5-items measure or trait optimism. The second set includes measures regarding an individual’s attachment to various non-empirical beliefs such as religiosity, narrative activities such as reading novels or watching movies, and political views. In order to check whether the abnormal circumstances of the Covid-19 epidemic may be affecting our results, we will also ask participants to indicate how closely they follow the Covid-19 epidemic and to guess the percentage of the population that has been infected by the virus in their state. The third set includes some demographic information including age, gender, education, the state they live in, employment status, household income, family structure (i.e. number of siblings and children), and marital status. These are all the individual level information that will be asked to be provided by the participants. At the end of the study, we will also ask participants to report how well they think they performed on this game relative to an average participant. All participants will then be provided with a feedback regarding their performance in terms of the accuracy of their final guess.
We will construct measures of the performance of individuals relative to 
a) what would be their optimal guess conditional on their observations, and 
b) the true proportion of balls in the urn. 
For each type of measure, we will measure bias (the difference between the guess and the appropriate value), accuracy (the absolute value of the bias) and convergence (the difference between accuracy on the first guess and the final guess). This will yield six measures in total:
1) Conditional bias – the difference between an individual’s final guess and their optimal guess conditional on their observations;
2) Unconditional bias – the difference between an individual’s final guess and the true value;
3) Conditional accuracy – the absolute value of the conditional bias;
4) Unconditional accuracy – the absolute value of the unconditional bias;
5) Conditional convergence – the difference between the conditional accuracy at the third round and at the first round;
6) Unconditional convergence – the difference between the unconditional accuracy at the third round and at the first round.
In large samples, the mean values of the conditional measures should converge to the mean values of the unconditional measures. 
We plan to test the following hypothesis in this experiment:
H1: Both conditional and unconditional bias are increasing functions of f(x) = f(0.5 – P) with f(0) = 0, where P is the true proportion of the balls of the relevant color in the urn.
H2: Individual characteristics have no direct impact on conditional or unconditional bias at f(0), but may affect the slope f’(x), with the slope decreasing in individual risk aversion and conscientiousness and increasing in openness to experience.
H3: Both conditional and unconditional accuracy are increasing functions of education, and decreasing functions of the participants’ attachment to non-empirical beliefs. 
H4: Both conditional and unconditional convergence are increasing functions of education, and decreasing functions of the participants’ attachment to non-empirical beliefs.
In addition, in case we find a correlation between participants' performance in the urn game and either how closely they follow the Covid-19 epidemic or how accurately they guess the percentage of the population infected in their own state, we expect this correlation to disappear after controlling for the participant’s level of education.
Our baseline specification will involve a set of linear ordinary least squares regressions predicting the six different dependent variables reported above. We will include the true value of the proportion of the balls in the urn, and a set of individual characteristics as independent variables.
We will also include an attention check question at the end of the study. Participants will only be included in the analysis if they pass this attention check.
Experiment 1 will be conducted online with a target sample size of 1000 participants that is representative of the U.S. population employed via the subject pool provided by Prolific. Participants will be paid a fixed rate of £1.25 (~$1.50) for their participation. The experiment will employ an incentive compatible design, such that we will offer a bonus payment of £5 (~$6) to the top 20 participants who make the most accurate statistical judgments in the urn task.