A Simple Sequential Algorithm for Approximating Bayesian Inference

A Simple Sequential Algorithm for Approximating Bayesian Inference Elizabeth Bonawitz, Stephanie Denison, Annie Chen, Alison Gopnik, & Thomas L. Griffiths {liz b, smdeniso, c annie, gopnik, tom griffiths}@berkeley.edu Department of Psychology, 5427 Tolman Hall Berkeley, CA 94720 USA Abstract People can apparently make surprisingly sophisticated in- ductive inferences, despite the fact that there are constraints on cognitive resources that would make performing exact Bayesian inference computationally intractable. What algo- rithms could they be using to make this possible? We show that a simple sequential algorithm, Win-Stay, Lose-Shift (WSLS), can be used to approximate Bayesian inference, and is consis- tent with human behavior on a causal learning task. This al- gorithm provides a new way to understand people’s judgments and a new efficient method for performing Bayesian inference. Keywords: Bayesian inference; algorithmic level; causal learning Introduction In the last five years a growing literature has demon- strated that people often act in ways consistent with optimal Bayesian models (e.g., Griffiths & Tenenbaum, 2005; Good- man, Tenenbaum, Feldman, & Griffiths, 2008). These ap- proaches have provided a precise framework for characteriz- ing intuitive theories and have provided an account of how a learner should update her beliefs as evidence is acquired (Gopnik & Schulz, 2004; Griffiths & Tenenbaum, 2009). The theory-based Bayesian approach has met with much success in describing the inferences made by adults (for a review see Tenenbaum, Griffiths, & Kemp, 2006) and research in cog- nitive development suggests that children can make similar inferences (Gopnik & Schulz, 2007; Gopnik et al., 2004; Gweon, Schulz, & Tenenbaum, 2010; Kushnir & Gopnik, 2007; Schulz, Bonawitz, & Griffiths, 2007). Taken together, this research strongly suggests that Bayesian statistics pro- vides a productive starting point for understanding human in- ductive inference. Theory-based Bayesian approaches have typically been used to give a “computational level” (Marr, 1982) analysis of the inferences people make when solving inductive prob- lems, focusing on the form of the computational problem and its ideal solution. However, it need not be the case that the algorithms people are using to solve these problems actually resemble exact Bayesian inference. Indeed, given the compu- tational complexity of exact Bayesian inference (Russell & Norvig, 2002) and the numerous findings that children and adults alike have difficulty with explicit hypothesis testing (e.g., Kuhn, 1989; Klahr, Fay, & Dunbar, 1993) and some- times only slowly progress from one belief to the next (Carey, 1991; Wellman, 1990), it becomes interesting to ask how learners might be behaving in a way that is consistent with Bayesian inference. Here we investigate the algorithms that learners might be using in solving a particular kind of inductive problem – causal learning. These algorithms need to approximate Bayesian inference, but also need to be computationally tractable. One strategy that has proven effective for approx- imating Bayesian inference in computer science and statis- tics is using sampling-based approximations, also known as Monte Carlo methods. We introduce a new sequential sam- pling algorithm based on the Win-Stay, Lose-Shift (WSLS) principle, in which a learner maintains a particular hypothe- sis until receiving evidence that is inconsistent with that hy- pothesis. We show that this WSLS algorithm approximates Bayesian inference, and can do so quite efficiently. Previous work in cognitive psychology has shown that peo- ple follow a WSLS strategy in concept learning tasks (Restle, 1962; Levine, 1975). We use this as the starting point for an investigation of whether human behavior that approximates Bayesian inference in causal learning might be explained in terms of a WSLS strategy. We compare the WSLS algo- rithm to simply sampling from the posterior distribution as an account of human behavior in a simple causal learning task. Both algorithms predict that people should behave in a way that is consistent with Bayesian inference, but WSLS also predicts that there should be a characteristic pattern of dependency between people’s successive responses. The plan of the paper is as follows. First, we introduce the causal learning task that will be the focus of our analysis, and summarize how Bayesian inference can be applied in this task. We then introduce the idea of sequential sampling algo- rithms, including our new WSLS algorithm. This is followed by a mathematical analysis of the WSLS algorithm, show- ing that it approximates Bayesian inference. The remainder of the paper focuses on an experiment in which we evaluate how well the WSLS algorithm captures people’s judgments in our causal learning task. Bayesian inference and causal learning While the algorithms that we present in this paper will apply to any inductive problem with a discrete hypothesis space, we will make our analysis concrete by focusing on a simple causal learning problem. In this problem, there are three cat- egories of objects: red, green, and blue blocks. Each of these kinds of blocks activate a machine with different probability when they are placed on the machine. The red blocks acti- vate the machine on five out of six trials, the green blocks on three out of six trials, and the blue blocks on just once out of six trials. A new block is then presented, which has lost its color, and needs to be classified as either a red, green, or blue block, based on some observations of what happens when it is placed on the machine over a series of trials. The question