# Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing
## 5 Imitation Learning-Based Control  
The ability to utilize expert behavior and learn about tasks from demonstrations is the basis of imitation learning [91]. Opposite to reinforcement learning, where the agent learns the robot control strategies by interacting with the environment, imitation learning makes use of demonstrated trajectories, which typically are represented by states or state-action pairs [67]. The two major classes of imitation learning are behavioral cloning and inverse reinforcement learning. Behavioral cloning is often referred to as learning a policy that directly maps from the state to the control input. Alternatively, recovering the reward function from demonstrations is referred to as inverse reinforcement learning [92]. The demonstrations can be acquired in several different direct and indirect ways such as teleoperation, kinesthetic teaching, motion capture, virtual/augmented reality and video demonstrations [93]. Such skill transfer can not only increase usability and cope with the absence of skilled industrial robot programmers on the production floor but also increase efficiency and achieve more human-like performance. The usage of the demonstration method typically depends on the details of the task; however, kinesthetic teaching illustrated in Figure 9[94; 95; 96] can be advantageous, as demonstrations are directly acquired from robot movements, which eliminate imprecision in the form of a noise, which can be introduced by the usage of supplementary sensors. A major limitation of kinesthetic teaching is the requirement of gravity compensation, which is not supported by all of the industrial robots. Another direct approach--teleoperation teaching--can be performed by different control input devices such as joystick [97], hand tracking hardware [98] and other remote control devices. In contrast to direct teaching, indirect teaching does not directly control the robot. Such systems typically learn from visual demonstrations [99] and w  
Figure 8: U shape object assembly [87]. Figure 9: Demonstration by kinesthetic teaching [94]. capturing human motions to generate trajectories for robots. However, such learning from visual representations lacks valuable information of robot interaction with the environment, therefore requiring additional extensive training. Thus, the difficulty of mimicking human behavior based on visual information is raised due to a comparatively complex demonstration acquisition process. In this manner, the actions performed by human needs to be recognized, interpreted and mapped into the imitator space. A self-supervised representation learning method based on multi-view video was proposed in [99]. The reward function for RL was created by using only raw video demonstrations for super-vision and human pose imitation without any explicit joint level correspondence. The experiments demonstrated effective learning of a pouring task with a real robot illustrated in Figure 10. The wearable device (tactile glove) used in [100] enabled observation of visually concealed changes of the workpiece. The human demonstration included both hand pose and contact forces to learn the task of opening child-safe medicine bottles. Even though indirect teaching is mostly addressed in tackling more of everyday situations that are applicable, for example, in elderly care [101], the used methods perfectly match the smart manufacturing challenges. Demonstrations can also be learned in a virtual environment by training a virtual robot, as depicted in Figure 11. Such virtual reality approach achieved an overall grasping success rate of 74% grasping fish in real-world settings by acquiring 35 virtual demonstrations, which were synthetically expanded to 100,000 grasping samples [102]. Thus, with a VR-based demonstration, the expert knowledge can be acquired remotely, without physical presence. Complex manipulations tasks were also learned in [98] by combining virtual reality and teleoperation teaching.