ROBotic Open-architecture Technology for Cognition, Understanding, and Behavior

control class<DEVICE, PARAMETERS> Locale definition class Virtual device driver Figure 9: The structure of a control class for a generic device. Experimental results of tests with existing platforms Development of a Cognitive Humanoid Cub Date: Version: No. Page 12 of 34 3 . 5 R o b o t s p e c i f i c i n t e r f a c e The real “communication” with the robot is carried out through a set of binary modules that use the device driver structure described in the previous section. Module customization is at this stage accomplished through configuration files. In the YARP language these modules are called daemons (a term borrowed from UNIX). The daemons directly interact with the remainder of the robot software through YARP ports and in general they export very specialized communication channels. For example the frame grabber has an output port of type “image” and the head control daemon an input port that accepts velocity commands. There are no specific restrictions on the type of ports exported by a daemon since any type of state information about the modules might be required. Further, some of the daemons accept or send commands of a special type that are generally used to communicate status information. A bus structure based on the MCAST protocol has been implemented to transmit and receive these special messages (called “bottles”). YARP bottles may contain any type of data or even a group of heterogeneous elements of different types. The structure contains identifiers to properly decode messages and interpret the data. YARP bottles create a network within the network of behaviors to realize a high-level control and coordinate a large number of modules. 3 . 6 L e a r n i n g a r c h i t e c t u r e † This layer describes an arrangement of YARP modules that tends to repeat across our robotic architecture. This is not formally into YARP proper but simply an implementation of a particular experiment relying on YARP libraries. Conceptually it forms a layer where to build more sophisticate experiments since for example it provides simple motor control and sensorimotor coordinative behaviors. Overall they could be seen as very high level commands that support positioning, gazing, reaching for visually identified objects, and grasping them. Grossly speaking, autonomous learning requires a slightly different approach from classical supervised paradigms where data is presegmented and simply fed into a function approximator. Autonomous learning is perhaps closer to reinforcement learning in that it requires action and proper behaviors (exploratory) to gather the training set. Necessarily our architecture will require bootstrapping behaviors supporting building the training set. The question of how much explore and how to get quickly to a solution is an open one in reinforcement learning and unfortunately reinforcement learning itself tend to be difficult, requiring a very large number of samples. In addition, in the case of a real robot we shouldn’t allow “spurious” or random control values to get to the low-level controllers; at the basis of any control strategy we should probably have a reasonable “safe” explorative procedure and certainly not a complete random one. Self-supervised procedures can be identified (similar in spirit to Kawato’s feedback error learning) and given the appropriate amount of exploration they can quickly approximate the desired sensorimotor coordination pattern. When data samples are available in sufficient number with respect to the size of the parameter space of the function approximator of choice the system can start learning † Can be skipped on a first read. Experimental results of tests with existing platforms Development of a Cognitive Humanoid Cub Date: Version: No. Page 13 of 34 and using what has been learnt up to date; necessarily in the long run the influence of explorative behaviors should be reduced. At least two possibilities exist here: learning could be implemented either in batches or fully online. The specific strategy is mostly a function of the algorithm and specific implementation of the function approximation. Inhibition or a functional equivalent should take care of reducing or mixing up exploration with actual “exploitation” of the acquired behavior. Our discussion is only focused here on the function approximation problem since a good part of the sensorimotor behaviors can be actually well implemented by mapping sensory values onto motor commands or the opposite or even by a combination of the two (e.g. feedback error learning or distal learning). Another constraint on the design of explorative behaviors is that they should mostly “explore” the space that will be used in the future. Needless to say that failure to do so might result in very poor performance. The learning algorithm can be conceptually divided in two parts: the one providing the “learning signals” sometimes called the “critic”, and the one doing the behavior called the “actor”. This distinction is important in motor control problems since the actor must be extremely fast and should work in a small delay regime. On the other hand, the critic could take even seconds or minutes to process the training data and provide infrequent adjustments to the actor’s parameters. We maintained as much as possible (apart from trivial cases) this distinction within our system. This division is to some extent compatible with biological mechanisms of learning being these for example the rates at which synaptic changes and growth processes develops in the brain compared to actual spikes’ travel times. Exploratory component