## 3 Our Approach
### Algorithm  
We here present the algorithm SGIM-PB (Socially Guided Intrinsic Motivation with Procedure Babbling). In Section 5.5, we will also look at the variant named SGIM-TL (Socially Guided Intrinsic Motivation with Transferred Lump), which is provided at initialisation with a dataset of transferred procedures and their corresponding reached outcomes: \(\{(\omega_{i},\omega_{j}),\omega_{r}\}\). The main differences between SGIM-PB and SGIM-TL are outlined and they are contrasted with former versions IM-PB and SGIM-ACTS in Table 1. SGIM-PB and SGIM-TL propose to learn both \(M\) and \(P\) simultaneously. Figure 2: Task hierarchy of the Yumi experimental setup : blue lines represent task decomposition into procedures for the simulation setup, dashed lines for the physical setup, red lines for the direct inverse model. Eg. to move a blue object placed initially at \((x_{0},y_{0})\) to a desired position \((x_{1},y_{1})\), the robot can carry out task (\(\omega_{i}\)) that consists in moving its end-effector to \((x_{0},y_{0})\) to pick it, then task (\(\omega_{j}\)) that consists in moving the end-effector to \((x_{1},y_{1})\). These subtasks are executed with respectively action primitives \(\pi^{i}\) and \(\pi^{i}\). Therefore to move the object, the learning agent can use the sequence of action primitives \((\pi^{i},\pi^{i})\). SGIM-PB starts learning from scratch. It is only provided with :  
* Dimensionality and boundaries of the action primitive space \(\Pi\);
* Dimensionality and boundaries of each of the outcome subspaces \(\Omega_{i}\subset\Omega\);
* Dimensionality and boundaries of the procedural spaces \((\Omega_{i},\Omega_{j})\subset\Omega^{2}\), defined as all possible pairs of two outcome subspaces. Our SGIM-PB agent is to collect data in order to learn how to reach all outcomes by generalising from the sampled data. This means it has to learn for all reachable outcomes, the actions or procedures to use to reach the outcome. This corresponds to learning the inverse model \(M\). The model uses a local regression based on the k-nearest neighbours from the data collected by exploration. In order to do that, the learning agent is provided with different exploration _strategies_ (see Section 3.3.1), that are defined as methods to generate a procedure or action for any given outcome. The 4 types of strategies available to the robot are: two autonomous exploration strategies and two interactive strategies per task type. For the autonomous exploration strategies, we consider action space exploration and procedural space exploration. For the interactive strategies, we consider mimicry of an action or a procedure teacher of the task type: the former's demonstrations are motor actions, while the latter's are procedures. As these strategies could be more appropriate for some tasks than others, and as their effectiveness can depend on the maturity of the learning process, our learning agent needs to map the outcome subspaces and regions to the best suited strategies to learn them. Thus, SGIM-PB uses an _Interest Mapping_ (see Section 3.3.2) that associates to each strategy and region of an outcome space partition an _interest_ measure, so as to guide the exploration by indicating which tasks are the most interesting to explore at the current learning stage, and which strategy is the most efficient. The SGIM-PB algorithm (see Algorithm 1, Figure 3) learns by episodes, and starts each episode by selecting an outcome \(\omega_{g}\in\Omega\) to focus on and an exploration strategy \(\sigma\). The strategy and outcome region are selected based on the Interest Mapping by roulette wheel selection also called fitness proportionate selection, where the interest measure serves as fitness (see Section 3.3.2). \begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
\begin{tabular}{} \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Differences between SGIM-PB, SGIM-TL, SGIM-ACTS and IM-PB.