Intelligent Adaptive Curiosity: a source of Self-Development

This paper presents the mechanism of Intelligent Adaptive Curiosity. This is a drive which pushes the robot towards situations in which it maximizes its learning progress. It makes the robot focus on situations which are neither too predictable nor too unpredictable. This mechanism is a source of selfdevelopment for the robot: the complexity of its activity autonomously increases. Indeed, we show that it first spends time in situations which are easy to learn, then shifts progressively its attention to situations of increasing difficulty, avoiding situations in which nothing can be learnt. 1. An engine for self-development Development involves the progressive increase of the complexity of the activities of an agent with an associated increase of its capabilities. This process is now recognized to be crucial for the formation of nontrivial intelligence (Brooks and Steins, 1994; Weng et al., 2001). Thus, one of the goals of developmental robotics is to build robots which develop. Several papers have already been written showing how one could increase the efficiency of the learning of a robot by putting it in situations in which the task to be solved was of increasing difficulty (e.g. Elman, 1993; Nagai et al., 2002). Yet, in all of them to our knowledge, the building of learning situations of progressive complexity was done manually by a human. As a consequence, the robots were not developing autonomously: they were involved in passive development. The goal of this paper is to present a mechanism which enables a robot to autonomously develop in a process that we call selfdevelopment (active development may also be used). This means that the robot is put in a complex, continuous, dynamic environment, and that it will be able to figure out by itself without prior knowledge which situations in this environment have a complexity which is suited for efficient learning at a given moment of its development. This mechanism is called Intelligent Adaptive Curiosity (IAC): • it is a drive in the same sense than food level maintenance or heat maintenance are drives, but it is much more complex. Instead of being about the maintenance of a physical variable, the IAC drive is about the maintenance of an abstract dynamic cognitive variable: the learning progress, which must be kept maximal. • it is called curiosity because maximizing the learning progress pushes the robot towards novel situations in which things can be learnt. • it is adaptive because the situations that are attractive change over time: indeed, once something is learnt, it will not provide learning progress anymore. • it is called intelligent because it keeps the robot away both from situations which are too predictable and from situations which are too unpredictable (i.e. the edge of order and chaos in the cognitive dynamics). Indeed, naive curiosity algorithms like “go in most novel or in most unpredictable situations” might lead the robot to completely chaotic situations which can be dangerous and where nothing can be learnt (e.g. bumping very fast into walls and bouncing against it). IAC is related to emotions and value systems (Sporns, 2000): indeed, it tags all situations as positive or negative with real numbers and accordingly to their learning progress potential, and makes the robot act so that it finds itself often in positive situations, which bring internal positive rewards. The next section will explain in detail and in a practical case the concept of Intelligent Adaptive Curiosity. This new system is more general and much more robust than the initial system we proposed in (Kaplan and Oudeyer, 2003), as well as the system of (Schmidhuber, 1991). 127 In Berthouze, L., Kozima, H., Prince, C. G., Sandini, G., Stojanov, G., Metta, G., and Balkenius, C. (Eds.)