Active learning strategies in a spatial concept learning game

Active Learning Strategies in a Spatial Concept Learning Game Todd M. Gureckis (todd.gureckis@nyu.edu) Doug Markant (doug.markant@nyu.edu) New York University, Department of Psychology 6 Washington Place, New York, NY 10003 USA Abstract In order to analyze participant’s performance in such a complex and dynamic learning task, we develop a formal model of information search based on Bayesian learning prin- ciples. The model specifies how past observations should in- fluence current beliefs, and how uncertainty should translate into information sampling behavior on a trial-by-trial basis. By comparing the utilities the model assigned to each possi- ble observation with the selections that participants make, we are able to characterize the efficiency of participants’ search strategies relative to an ideal learner who had experienced the same set of previous observations. In addition, the model al- lowed us to objectively classify particular information collec- tion decisions as being either “exploitative” of known contin- gencies or “exploratory” of relatively unknown parts of the game board. We find interesting behavioral differences be- tween these two modes of information search. Effective learning often involves actively querying the envi- ronment for information that disambiguates potential hypothe- ses. However, the space of observations available in any situ- ation can vary greatly in potential “informativeness. ” In this report, we study participants’ ability to gauge the informa- tion value of potential observations in a cognitive search task based on the children’s game Battleship. Participants selected observations to disambiguate between a large number of po- tential game configurations subject to information-collection costs and penalties for making errors in a test phase. An “ideal- learner” model is developed to quantify the utility of possi- ble observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient “exploitive” mode of local search and a faster, less efficient pattern of “exploration. ” Traditional experimental approaches to human learning tend to emphasize passive learning situations. For example, in a typical concept learning task, subjects are presented with examples one at a time, the order of which are selected by the experimenter (often at random and with exhaustive sam- pling of the training set). However, this procedure ignores the fact that real-world learning often requires learners to ac- tively create their own learning experiences by constructing revealing queries or engaging in exploration of unknown con- tingencies (Nelson, 2005; Skov & Sherman, 1986; Sutton & Barto, 1998). For example, children might ask about partic- ular objects in their environment and receive feedback from an adult (e.g., “What is that? ”, “What does that do? ”). In or- der for such sampling behavior to be effective, queries should be directed to maximize the potential information that could be obtained from an answer. We need not ask about things that we already know, and, all else being equal, should pre- fer questions whose answers are expected to be most reveal- ing (Klayman & Ha, 1987; Nelson, 2005; Oaksford & Chater, In this paper, we present an initial study examining the mu- tual unfolding of information search and learning in a task modeled on the classic children’s game Battleship. Partici- pants attempted to learn a hidden “concept” (the shape and spatial configuration of three hidden rectangles) by sequen- tially uncovering points on a large grid (see Figure 1). Each observation cost points and participants were motivated to minimize the points accumulated in each game. As a result, they had to minimize the number of observations they made in order to correctly identify the hidden rectangle configura- tion. Active Sampling in Concept Acquisition The ultimate goal of the present work is to understand how people actively seek information when acquiring new con- cepts. Previous work has shown that allowing learners to make their own decisions about what information to sample can have an impact on both the efficiency of learning (Castro et al., 2008) and what is learned (Fazio, Eiser, & Shook, 2004) in concept acquisition tasks. For example, Castro, et al. (2008) found that allowing learners to actively select train- ing examples greatly improved the efficiency by which they learned a linear decision boundary. Fazio, Eiser, and Shook (2004) studied the impact of experiential sampling on cat- egory learning. Participants were presented with different “beans” and were asked on each trial if they would like to (virtually) eat the bean and find out if it was healthy or poi- sonous. Decisions to not eat a particular bean thus provided no information (i.e., feedback in the task was contingent on sampling decisions). Experiential learners were found to be risk averse, in that they showed a bias to think that novel beans were bad and were more accurate classifying bad beans than good beans. Interestingly, this asymmetry in learning only occurred in situations where learners made the sampling decisions themselves as opposed to conditions where full in- formation was provided on each trial. While allowing learners to make decisions about what they want to learn about can have interesting consequences for what is learned, these studies leave aside the question of ex- actly how people decide which observations to make. How- ever, assessing the “usefulness” of potential observations has a long history of study in psychology, particularly in hypoth- esis testing situations (Skov & Sherman, 1986; Klayman &