## 2 Learning, Improving, and Generalizing Motor Skills Based on Imitation Learning and Self-Learning
### Improving and Generalizing Motor Skills through RL
We distinguish the improvement and generalization processes as follows: improvement updates the policy parameters using the RL process without changing the target of its DMP, whereas generalization adds new parameter tuples after modifying and updating the policy parameters and target of a DMP. These processes are determined by the HMMs and TM. The generalization process is selected when the likelihood of the TM is higher than those of all other HMMs, and the improvement process is selected when the likelihood of one of the HMMs is higher than that of the TM. We evaluated these processes by applying them to different peg and hole shapes. The \"hole search\" and \"peg insertion\" motor skills learned for the rectangle shape were generalized for triangle, pentagon, hexagon, and star shapes. These algorithm and reward functions improved the paths of the initial motor skills and optimized them to reduce the number of execution time steps. In the future, we will analyze the manner in which various humans perform peg-in-hole tasks and compare the reward functions with those learned through inverse RL to enable interpretation. We will verify our framework by means of industrial applications and various other robotic tasks. In addition, we will propose a method to determine the information to be noted in the human demonstrations and use them in modeling motor skills and reward functions. The Supplementary Materials are available online at [[http://www.mdpi.com/2076-3417/10/8/2719/s1](http://www.mdpi.com/2076-3417/10/8/2719/s1)]([http://www.mdpi.com/2076-3417/10/8/2719/s1](http://www.mdpi.com/2076-3417/10/8/2719/s1)). Conceptualization, S.H.L. and I.H.S. ; methodology, S.H.L. and N.J.C. ; software, S.H.L. and N.J.C. ; validation, N.J.C., J.B.K., and I.H.S. ; formal analysis, S.H.L. and N.J.C. ; investigation, S.H.L. ; resources, N.J.C. ; data curation, N.J.C. ; writing--original draft preparation, S.H.L. ; writing--review and editing, N.J.C. and J.B.K. ; visualization, S.H.L. and N.J.C. ; supervision, I.H.S. ; project administration, I.H.S. ; and funding acquisition, S.H.L. All authors have read and agreed to the published version of the manuscript. This research received no external funding. This work was supported by the Technology Innovation Industrial Program funded by the Ministry of Trade, (MI, South Korea) (10073161), Technology Innovation Program. This work also has been conducted with the support of the Korea Institute of Industrial Technology as \"Development of holonic manufacturing, system for future industrial environment(KITECH EO-20-0019)\". Finally, this work was supported by the Institute for Information&communications Technology Promotion (IITP) grant funded by MSIT (No. 2018-0-00622). The authors declare no conflict of interest. The funders had no role in the design of the study, the collection, the analyses, the interpretation of data, the writing of the manuscript, and the decision to publish the results. ## References

* Kronander et al. (2014) Kronander, K.; Burdet, E.; Billard, A. Task Transfer via Collaborative Manipulation for Insertion Assembly. In Proceedings of the Workshop on Human-Robot Interaction for Industrial Manufacturing, Robotics, Science and Systems, Bielefeld, Germany, 3-6 March 2014; pp. 1-6. * Billard et al. (2016) Billard, A.; Calinon, S.; Dillmann, R. Learning from Demonstration. In _Springer Handbook of Robotics_; Springer: Berlin, Germany, 2016; pp. 1995-2014. * Mollard et al. (2015) Mollard, Y.; Munzer, T.; Thibaut, B.; Baisero, A.; Toussaint, M.; Manuel, M. Robot Programming from Demonstration, Feedback and Transfer. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September-2 October 2015; pp. 1825-1831. * Gupta et al. (2016) Gupta, A.; Eppner, C.; Levine, S.; Abbeel, P. Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrations. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9-14 October 2016; pp. 3786-3793.