Constraining Bayesian Inference with Cognitive Architectures: An Updated Associative Learning Mechanism in ACT-R.

Constraining Bayesian Inference with Cognitive Architectures: An Updated Associative Learning Mechanism in ACT-R Robert Thomson (thomsonr@andrew.cmu.edu) Department of Psychology, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA, 15213, USA Christian Lebiere (cl@cmu.edu) Department of Psychology, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA, 15213, USA Abstract Bayesian inference has been shown to be an efficient mechanism for describing models of learning; however, concerns over a lack of constraint in Bayesian models (e.g., Jones & Love, 2011) has limited their influence as being a description of the ‘real’ processes of human cognition. In this paper, we review some of these concerns and argue that cognitive architectures can address these concerns by constraining the hypothesis space of Bayesian models and providing a biologically-plausible mechanism for setting priors and performing inference. This is done in the context of the ACT-R functional cognitive architecture (Anderson & Lebiere, 1998), whose sub-symbolic information processing is essentially Bayesian. To that end, our focus in this paper is on an updated associative learning mechanism for ACT-R that implements the constraints of Hebbian-inspired learning in a Bayesian-compatible framework. Keywords: cognitive architectures; Bayesian inference; Hebbian learning; cognitive models; associative learning; Introduction Bayesian approaches to reasoning and learning have been successful in such fields as decision-making (Tenenbaum, Griffiths, & Kemp, 2006), language learning (Xu & Tenenbaum, 2007), and perception (Yuille & Kersten, 2006). Most specifically, Bayesian inference has been exceptional in discovering some of the structure of language and word learning with substantially less training than traditional connectionist networks. Despite their successes, Bayesian models have come under attack for being unconstrained, unfalsifiable, and overly reliant on optimality as an assumption for reasoning (see Jones & Love, 2011; Bowers & Davis, 2012 for an exhaustive review; and Griffiths et al., 2012 for a counter- argument). While these criticisms are not without merit (nor are the Bayesians’ rebuttals fully convincing), the issue of constraints remains a critical argument. It is also not a new argument. Over 25 years ago the constraint argument was leveled against the field of connectionism (Fodor & Pylyshyn, 1988). Then it was argued that, via several learning rules and organizing principles, any behavior could theoretically be captured by connectionist networks. The degree that progress has slowed for the explanatory power of connectionist networks is beyond the scope of this paper; however, constraints on neural network development using a common learning rule in a stable cognitively- plausible architecture have been advanced (O’Reilly, 1998; O’Reilly, Hazy, & Herd, 2012). By corollary, to address similar concerns, the Bayesian movement needs to develop constraints which balance the computational transparency of their models with algorithmic and implementation (i.e., neural) level cognitive plausibility. Interestingly, ACT-R 6.0 (Anderson et al., 2004) is a cognitive architecture which already uses Bayesian-inspired inference to drive sub-symbolic learning (i.e., to generate and update the activation strength of chunks in declarative memory). The architecture is both constrained by learning rules (e.g., activation equations; base-level learning) and neuro-cognitively justified by many studies (Anderson & Lebiere, 1998; Anderson et al., 2004; Anderson, 2007). While there have been difficulties in adapting some aspects of the Bayesian approach (e.g., in implementations of associative learning), ACT-R serves as an example whereby Bayesian inference can be constrained by a neurally- localized and behaviorally-justified cognitive architecture. In this sense, ACT-R can act as a bridge between all three layers of Marr’s tri-level hypothesis. For the remainder of this paper, we present an overview of the debate over the applicability of Bayes inference to cognition and argue that ACT-R represents the kind of constraint that addresses criticisms against Bayesian models. We will further describe an updated associative learning mechanism for ACT-R that links Bayesian-compatible inference with a Hebbian-inspired learning rule. Bayesian Inference The essential feature of Bayesian inference is that it reasons over uncertain hypotheses (H) in probability space (i.e., from 0 – 100% certainty). The Bayes rule is defined as: where the posterior probability of an outcome is derived from the likelihood of the hypothesis explaining the data, combined with the prior probability of the hypothesis , and normalized by the probability of the data P(D). Thus, updating one’s belief is based on one’s prior belief influenced by the likelihood that some new evidence supports this belief. At its core, Bayesian inference is an excellent derivation of the scientific method. A difference between Bayesian models and connectionist implementations is that Bayes models of human cognition tend to use richer, more structured, and symbolic knowledge than connectionist models, which tend to use more distributed representations operating over less structured input. This level of inference places Bayesian models at the computational level of Marr’s tri-level hypothesis, whereas cognitive architectures and connectionist networks operate