A Curious Robot Learner for Interactive Goal-Babbling: Presentation
Channel: Nguyen Sao Mai
Date: 01/09/2017

PhD Defense, 2013.

The pdf manuscript is available from : http://nguyensmai.free.fr/thesislink.html
See the video of the slides on https://youtu.be/mI_14kzpGHQ

*****

Shortcuts to the sections of the video:
1. 1:42  Background : Life-Long Learning 
    (chapter 1.2 of the pdf)
2. 3:14  Challenge : Sampling an Open-ended Environment 
    (chapter 1.3 of the pdf)
3. 12:46  Idea : Data-collection Strategy based on Social Guidance and Artificial Curiosity 
    (chapter 2 of the pdf)
4. 22:15 Realisation1 : Algorithmic Architecture to Learn Faster and with Better Precision
    (chapter 3 of the pdf)
5. 41:41 Realisation2 : Modeling and Understanding Child Development 
    (chapter 5 of the pdf)
6. 44:06 Conclusion: Discover the Environment to Structure its Development 
    (chapter 6 of the pdf)

*****
Abstract:
The challenges posed by  robots operating in human environments on a daily basis and in the long-term point out the importance of adaptivity to changes which can be unforeseen at design time. The robot must learn continuously in an open-ended, non-stationary and high dimensional space. It must be able to know which parts to sample and what kind of skills are interesting to learn.  One way is to decide what to explore by oneself. Another way is to refer to a mentor. We name these two ways of collecting data "sampling modes". The first sampling mode correspond to algorithms  developed in the literature in order to autonomously drive the robot in interesting parts of the environment or useful kinds of skills. Such algorithms are called "artificial curiosity" or "intrinsic motivation" algorithms.  The second sampling mode  correspond to "social guidance" or "imitation" where the teacher indicates where to explore as well as where not to explore. Starting from the study of the relationships between these two concurrent methods, we ended up building  an algorithmic architecture with a hierarchical learning structure, called "Socially Guided Intrinsic Motivation" (SGIM).


We have built an intrinsically motivated active learner which learns how its actions can produce varied consequences or outcomes.  It actively learns online by sampling data which it chooses by using several sampling modes. On the meta-level, it actively learns which data collection strategy is most efficient for improving its competence and generalising from its experience to a wide variety of outcomes. The interactive learner thus learns multiple tasks in a structured manner, discovering by itself developmental sequences.

The study of the bootstrapping of autonomous exploration by social interaction with numerous users has led to algorithms such as :

    SGIM-D (Socially Guided Intrinsic Motivation by Demonstration) which uses imitation learning and actively chooses which outcome to learn to produce based on intrinsic motivation. See Nguyen, Baranes and Oudeyer, 2011, ICDL.
    SGIM-IM (Socially Guided Intrinsic Motivation with Interactive learning at the Meta level) which in addition actively chooses between an autonomous learning strategy or an imitation learning strategy. See Nguyen and Oudeyer, 2012, Humanoids.
    SGIM-ACTS (Socially Guided Intrinsic Motivation with Active Choice of Teacher and Strategy) which chooses actively and hierarchichally at each learning episode what and how to learn, and when, what and who to imitate, depending on measures of competence progress that each learning strategy and teacher induces for different kinds of outcome. See Nguyen and Oudeyer, Paladyn Journal of Behavioral Robotics, DOI 10.2478/s13230-013-0110-z.

*****
References

PhD Thesis (2013): A Curious Robot Learner for Interactive Goal-Babbling : Strategically Choosing What, How, When and from Whom to Learn. 
http://nguyensmai.free.fr/thesislink.html

Sao Mai Nguyen and Pierre-Yves Oudeyer (2012), Active Choice of Teachers, Learning Strategies and Goals for a Socially Guided Intrinsic Motivation Learner, Paladyn Journal of Behavioral Robotics, 3(3):136--146. 

Sao Mai Nguyen and Pierre-Yves Oudeyer (2014), Socially Guided Intrinsic Motivation for Robot Learning of Motor Skills, Autonomous Robots, doi:http://dx.doi.org/10.1007/s10514-013-9339-y, 36(3):273-294.