Empirical Evidence for Markov Chain Monte Carlo in Memory Search

Empirical Evidence for Markov Chain Monte Carlo in Memory Search David D. Bourgin (ddbourgin@berkeley.edu) Joshua T. Abbott (joshua.abbott@berkeley.edu) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology University of California Berkeley Berkeley, CA 94720 USA Abstract Kevin A. Smith (k2smith@ucsd.edu) Edward Vul (evul@ucsd.edu) Department of Psychology University of California San Diego La Jolla, CA 92093 USA mote Associates Test (RAT; S. A. Mednick, 1962), a classic multiply-constrained search task. Building on recent work using random walks on semantic networks to model semantic fluency (Griffiths, Steyvers, & Firl, 2007) and singly-constrained search in memory (Abbott, Austerweil, & Griffiths, 2012), we consider a range of MCMC-inspired models for search on the RAT. We find that human response patterns are well-matched by a model closely related to the Metropolis-Hastings (M-H) algorithm, provid- ing empirical support for MCMC as a candidate for modeling creative ideation. The plan for the paper is as follows. We first review pre- vious work on human response patterns on the Remote Asso- ciates Test and discuss existing theories of search on this task. We then introduce Markov chain Monte Carlo methods with an emphasis on their application to creative search. We end by formally introducing the model used in the current study, and offer an analysis of its behavior in relation to human data collected by Smith, Huber, and Vul (2013). Previous theoretical work has proposed the use of Markov chain Monte Carlo as a model of exploratory search in mem- ory. In the current study we introduce such a model and eval- uate it on a semantic network against human performance on the Remote Associates Test (RAT), a commonly used creativ- ity metric. We find that a family of search models closely re- sembling the Metropolis-Hastings algorithm is capable of re- producing many of the response patterns evident when human participants are asked to report their intermediate guesses on a RAT problem. In particular we find that when run our model produces the same response clustering patterns, local depen- dencies, undirected search trajectories, and low associative hi- erarchies witnessed in human responses. Keywords: Creativity; Remote Associates Test; Information retrieval; Semantic networks; Markov chain Monte Carlo. Introduction Exploratory search in memory is a major component of cre- ative problem solving. Often, the demands of a task impose constraints on the solution space: when searching for a word to rhyme with A that also means B, a would-be poet is un- likely to include items that mean T or that rhyme with Q (as- suming that Q does not rhyme with A and T is not a synonym of B) in her search. Similarly, an inventor would be remiss to consider inventions that do not satisfy certain requirements (e.g., novelty, usefulness) if she plans to pursue a patent. The current study proposes a formal model for the process by which people perform this type of exploratory search under multiple constraints. Locating relevant pieces of information in memory re- quires a strategy for quickly traversing the space of poten- tial solutions. Markov chain Monte Carlo (MCMC) methods have been a particularly useful in this regard, offering effi- cient statistical techniques for exploring spaces that would otherwise prove computationally intractable to traverse fully (Brooks, 1998). MCMC methods have been employed to model phenomena as diverse as theory change (Ullman, Goodman, & Tenenbaum, 2012), perceptual multistability (Gershman, Vul, & Tenenbaum, 2012), and conceptual flu- idity (Gabora, 2000), making them a popular tool within sta- tistical and computational cognitive modeling. MCMC meth- ods have also been proposed within the creativity literature to model search behavior in memory (e.g., Martindale, 1995; Paulus, Levine, Brown, Minai, & Doboli, 2010), although to date there exists little empirical evidence on which to evaluate these proposals. In the current study we aim to fill this gap by evaluating a formal model of MCMC search on the Re- Background Stage models of problem solving propose that individuals first search through memory to identify potential answers to a problem and then test those candidate answers against the constraints of the problem to determine acceptability (Gruenewald & Lockhead, 1980; Raaijmakers & Shiffrin, 1981). In the current paper it is assumed that these search and test processes constitute qualitatively distinct cognitive tasks. Thus in reviewing recent work on the RAT and mem- ory search we emphasize accounts that focus not only on the identification of correct answers but also on the path by which these answers are selected. The Remote Associates Test In the Remote Associates Test participants are shown a set of three cue words (e.g., ‘surprise,’ ‘line,’ ‘birthday’) and in- structed to generate a fourth word to relate them (in this case, ‘party’). Although originally introduced to test for individual differences creative ability (S. A. Mednick, 1962), early work established that performance on the RAT correlated with IQ (M. T. Mednick & Andrews, 1967) and originality during brainstorming (Forbach & Evans, 1981). More recently the RAT has also been used to measure the effects of manipu- lations related to creative ability, including intuition and in- cubation (Bowers, Regehr, Balthazard, & Parker, 1990; Vul & Pashler, 2007), the role of affect during problem solving