From Reactive Behaviour to Adaptive Behaviour: Motivational Models for Behaviour in Animals and Robo

From Reactive Behaviour to Adaptive Behaviour Motivational models for behaviour in animals and robots E. H. Spier, Balliol College, Trinity Term 1997 A thesis submitted for the degree of Doctor of Philosophy This thesis presents one possible way to design a control architecture that can be used to govern arti cial animals. Such artefacts perform multiple-tasks and are expected to exist in a somewhat hostile environment { they have to be adaptive. It also defends the position that automata, and animals, need not use reasoning to perform intelligent behaviour. Drawing from an ethological conception of motivation, a mathematical framework was described, computer simulations performed and preliminary work on a real robot discussed. It was shown that a reactive motivational algorithm performs better than alternatives that use simplistic models of the world, in a multiple resource foraging task. The reactive motivational framework was then extended to encompass instrumental behaviour as well as purely consummatory behaviour { an instantiation of the model was then demonstrated to exhibit complex sequences of behaviour that could give the impression of plan following, although there was none. This model permits us to introduce a useful technical de nition of the word tool. The model was then used to provide a fully procedural account of the outcome devaluation e ect { a phenomenon for which there is dispute over its cognitive status. The incentive learning e ect was also considered and found to be lacking any adequate explanation, in this work or in others. I still view science not as a search for imperishable truth but more as a kind of game. We play the game partly for the fun of it, but partly also because we believe it will increase our understanding. R.C. Bolles. Theory of motivation, second edition [Preface]. Harper & Row, NY. 1975. Behavior is always purposive action, or a train or sequence of purposive actions. W. McDougall. Outline of Psychology [Chapter 2]. Scribners, NY. 1923. It is the business of behavioristic psychology to be able to predict and control human activity. To do this it must gather scienti c data by experimental methods. Only then can the trained behaviorist predict, given the stimulus, what reaction will take place; or, given the reaction, state what the situation or stimulus is that has caused the reaction. J.B. Watson. Behaviorism, second edition [Chapter 1]. University of Chicago Press. 1930. Acknowledgements Acknowledgements are peculiar things. Properly, here, they come rst { for some may say that the contents of this book is what is human, however I prefer to think that these people are. Firstly, my supervisor, David McFarland; without him this thesis would simply not have existed { he suggested the project and has continually supplied insights while expecting me to think and communicate clearly. Next, Robin McCleery and Steve Simpson, my supervisory committee, have always welcomed my visits. I am also in the debt of my colleagues, friends and sometime compatriots: Claire Berlinski, Gidon Cohen, Jo-Anna Coles, Marian Dawkins, Rick D'Eath, Rupert DeBochgrave, Mary Eaves, James Erskine, James Fotheringham, Owen Holland, Kathyrn James, Elleanor Jupp, Alex Kacelnik, Sonia Khan, David Krakauer, Angus MacDougall, Niall Maclean, Georgia Mason, Candy Rowe, Steve Rogers, Peter Stuer, Adrian Thomas and Deborah Wexler. Whatever breadth of vision I possess in the actual pursuit of science was aided by welcoming stays at the laboratories of Rolf Pfeifer and Luc Steels. Finally, I would like to thank my Parents for the immensely supportive rôle they played, and have always played; in addition to their nancial assistance and e cient proof reading. Likewise, David Spier has also lifted much weight from this task I set myself. And, both my brother and sister have lovingly been just that.