A Critique of the Small Sample Account of Covariation Detection

A Critique of the Small Sample Account of Covariation Detection Wolfgang Gaissmaier (gaissmaier@mpib-berlin.mpg.de) Lael Schooler (schooler@mpib-berlin.mpg.de) Jorg Rieskamp (rieskamp@mpib-berlin.mpg.de) Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development Lentzeallee 94, 14195 Berlin, Germany The starting point for this project was the finding that people with a low working memory capacity perform better in a covariation detection task (Kareev, Lieberman, & Lev, 1997). In the task people successively encountered envelopes with two different colors and each time had to decide which out of two objects they think they will contain. The explanation for the low capacity advantage is that people with a lower working memory capacity have to rely on smaller samples when they make decisions. This helps them to detect a correlation earlier, because statistically small samples are more likely to indicate a correlation that exceeds the correlation in the population (Kareev, 1995b). Experiments We conducted two experiments with an extended version of the original task to test and model the original finding that low capacity people perform better in a covariation task. An implication of the small sample account is that they are also better in detecting a change in the correlational structure of the environment. As in the original experiment, working memory capacity was assessed with a digit span test. The original finding was replicated in the first but not in the second experiment, thus it seems to be a weak and unstable effect. It is worth noting that the probability of replicating a result at the same or a higher level of significance (and in the same direction) is only 50% (Goodman, 1992). Contrary to the predictions by the small sample account there was a high capacity advantage after a change in the first experiment. In the second experiment we did not find any differences between low and high capacity people, neither before nor after a change. Therefore, we focus on the first experiment with regard to modeling. Modeling Two different models have been tested, a naive window model and a reinforcement learning model. Every model was fitted to each individual separately since we wanted to relate capacity to model parameters. The naive window model that tries to translate the small sample idea directly could not capture the low capacity advantage. But we were able to model it with the reinforcement learning model (Camerer & Ho, 1999) with a decay, a sensitivity and an initial attraction parameter, where we forced the variance in each of the parameters separately by fixing the other two to their means. All three versions were able to capture the low capacity advantage on covariation detection, but only the initial attraction version was related to capacity and could predict behavior after a change. Conclusions The small sample account is not clearly supported by our data. First, the deduced hypothesis of a low capacity advantage after a change does not hold, we find either no effect or the opposite. Second, the naive window model and the reinforcement learning model version with the decay parameter which has the strongest connection to memory have to be rejected. Instead, an initial attraction parameter model is successful, indicating a faster learning process of low capacity people in the beginning, but not later on. Still, faster learning can be interpreted as relying on smaller samples. But it is also congruent with the finding of Weir (1964) that children use the simple but most successful payoff maximization strategy (i.e. always choose the more frequent option given a color) earlier in a similar task because they are simply reinforcement driven. Adults, in contrast, develop complex hypothesis and apply complex strategies because they believe that there exists a perfect solution, but they end up worse. As capacity differs between children and adults (Kail, 1984) and plays an important role in hypothesis generation (Dougherty and Hunter, 2003) this could be an explanation for the low capacity advantage. References Camerer, C.F. , & Ho, T. (1999). Experience-weighted attraction learning in normal-form games. Econometrica, Dougherty, M. R. P., & Hunter, J. E. (2003). Hypothesis generation, probability judgment and individual differences in working memory capacity. Acta Psychologica, 113, 263-282. Goodman, S.N. (1992). A comment on p-values, replication and evidence, Statistics in Medicine, 11, 875-879. Kail, R.V. (1984). The development of memory in children (2 nd ed.). New York: Freeman. Kareev, Y. (1995b). Through a narrow window: Working memory capacity and the detection of covariation. Cognition, 56, 263-269. Kareev, Y. , Lieberman, I. , & Lev, M. (1997). Through a narrow window: Sample size and the perception of correlation. Journal of Experimental Psychology: General, 126, 278-287. Weir, M. (1964). Developmental Changes in Problem- Solving Strategies. Psychological Review, 71, 473-490.