Mechanisms of Action Selection: Introduction to the Special Issue

Welcome to the special issue on Mechanisms of Action Selection. In this short introductory article, I provide a brief introduction to action selection and its history in the adaptive behavior community. Then, I introduce each of the articles in this issue. Finally, I acknowledge the many researchers who have contributed to this issue besides the present authors. Action selection is the process by which any agent chooses at any instant what to do next. I use here the term “agent” in the philosophical sense – as any actor capable of autonomously changing the world. Action selection has been heavily researched both in biology and in the adaptive behavior community. There are two components of the problem: first, deciding what an action is (i.e. what are the available options for selection), and second, determining which action to conduct at any particular moment. Initially in artificial intelligence (AI), it was taken for granted that action selection was simply a matter of executing the steps of a plan which would lead you from your current state to one of your goals. This plan was presumed to be a provably optimal one, formed by a formal process of search, or perhaps one acquired as a case from imitation learning or a set of social norms. In the mid-1980s, Chapman (1987) proved that discovering formally correct plans was impossible for resourcebounded agents, and in the mid-1990s, Wolpert (1996) did the same for learning. Meanwhile, Brooks (1986, 1991) led a shift of emphasis towards hand-coded systems with minimal on-board search. At this point, action selection took on a new emphasis for the community of researchers interested in real-time, animallike intelligence. Brooks and others (Maes, 1991; Malcolm & Smithers, 1990; Minsky, 1985; Parnas, Clements, & Weiss, 1985) emphasized the importance of modular decomposition for making the problem of intelligence tractable. Action selection for these systems became a problem not of planning but of integration – how could a disparate, distributed, parallel system behave in a coherent manner? In both natural and artificial intelligence, even this decomposition between primitives and control has been controversial (Prescott, Bryson, & Seth, 2007). Some researchers do not consider actions discrete, and/or do not believe they can be separated from their application. However, there is copious evidence from both fields that at least some aspects of intelligence really do work in this way. From neuroscience we have learned that discrete, complex actions can be both generated and recognized by simple systems involving individual nerve cells (Giszter, McIntyre, & Bizzi, 1989; Graziano & Gross, 1998; Rizzolatti, Fogassi, & Gallese, 2000). This is not to say that all behavior is controlled in this way – in vertebrates, at least, the outputs of these mechanisms seem to be smoothed by more continuous and dynamic control, as provided, for example, by the cerebellum. Two of the articles in this issue, those by Prescott and Davelaar, address the issue of identifying natural action-selection mechanisms. As computer scientists, AI researchers know that more homogeneous and continuous methods of control are, in theory, computationally equivalent to discrete and hierarchical ones. In practice, however, we have seen that exploiting modularity for coding (describing) behavior and providing specific action-selection mechanisms simplifies the design process and thus makes creating intelligence more probable (Bryson, 2000). In fact, by 1998, two significant publications had declared