# Object Detection-Based One-Shot Imitation Learning with an RGB-D Camera
## 3 Methodology  
The goal of the proposed method is enabling the robot to effectively interact with new, unknown objects in new environment from a single visual demonstration. In a BC framework, a policy \(\pi(\mathbf{a}\big{|}\mathbf{o})\) that maps observations \(\mathbf{o}\) to actions \(\mathbf{a}\) is learned with demonstrations generated by expert policies \(\pi^{*}\). A demonstration trajectory is composed of a series of observations and actions, which is formalized as \(\tau=[(\mathbf{o_{1}},\mathbf{a_{1}}),\dots,(\mathbf{o_{N}},\mathbf{a_{N}})]\). Trajectories of a task is formalized as \(\Gamma^{i}=[\tau_{1},\dots,\tau_{K}]\) and trajectories of different tasks \([\Gamma^{1},\dots,\Gamma^{L}]\) are collected for training the policy. To cope with new, unknown objects, TecNets [5] applied a task-embedding processing to find the similarity with the first frame \(\mathbf{o_{1}^{d}}\) and last frame \(\mathbf{o_{N}^{d}}\) of the new demonstration. The embedded vector was called as a sentence \(\mathbf{s}\big{(}\mathbf{o_{1}^{d}},\mathbf{o_{N}^{d}}\big{)}\) and the policy generated different actions based on the task sentence, which is modulated as \(\pi(\mathbf{a}\big{|}\mathbf{o},\mathbf{s})\). Following this pipeline, the semantic keypoint \(\mathbf{I}\) is applied to transfer specific intentions and the visuomotor policy is formalized as \(\pi(\mathbf{a}\big{|}\mathbf{o},\mathbf{I})\) where \(\mathbf{I}\big{(}\mathbf{o},\mathbf{o_{1}^{d}},\mathbf{o_{N}^{d}}\big{)}\) is a function of observations \(\mathbf{o}\), the first frame \(\mathbf{o_{1}^{d}}\) and last frame \(\mathbf{o_{N}^{d}}\) of the new demonstration. Because of the good generalization performance, depth map \(\mathbf{s}\)\(\mathbf{d}\) are used for the visuomotor policy and RGB images \(\mathbf{o}\) are applied for keypoint detection. The visuomotor policy is formalized as \(\pi(\mathbf{a}\big{|}\mathbf{d},\mathbf{I})\) where \(\mathbf{I}\) is an abbreviation of \(\mathbf{I}\big{(}\mathbf{o},\mathbf{o_{1}^{d}},\mathbf{o_{N}^{d}}\big{)}\). Any other trajectory in the same task can be regarded as the target demonstration in the training phase. As a result, the robot could effectively interact with new, unknown objects in new environment from a single visual demonstration in the test phase.  
Detailed structure of the proposed object detection-based one-shot imitation learning framework is shown in Figure 1. It mainly includes two parts: the object detection networks \(\mathbf{I}\big{(}\mathbf{o},\mathbf{o_{1}^{d}},\mathbf{o_{N}^{d}}\big{)}\) and the motion policy networks \(\pi(\mathbf{a}\big{|}\mathbf{d},\mathbf{I})\). The perception data in robotic applications is normally various and high dimensional. Convolutional neural network (CNN) is often used to process this perception information. For both RGB images and depth maps, CNN-based encoder networks are used to extract image features. These encoder networks are often pretrained to improve of the learning speed of the robot learning system. In this framework, they are trained with an autoencoder (AE) structure firstly and then fixed in the robot learning phase. Object detection networks and motion policy networks are trained separately. The trained object detection networks output the center of the task-related container, which is regarded as the semantic keypoint. The policy networks integrate the compressed depth feature blocks and the semantic keypoint to generate motion action. The structure of the autoencoder networks and other two networks are introduced successively.  
Figure 1: The main framework of the proposed method.