# Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation
## 2 Literature Review
### Industrial Applications of Object Pose Estimation  
The 3D pose estimation of objects has several industrial applications, and a bin picking technique is one of the examples. Bin picking is an essential technology for robots to pick up objects or parts and move them to another location or assemble them. Recent technology trends in pose estimation have been identified in the paper by [PERSON] et al. [1]. Studies on bin picking technology have been ongoing for a long time, even prior to recent deep-learning technology. Research on bin picking technology began with a study that used the feature information of objects in images. Pose estimation using image feature information extracts the feature information of objects detected in the input image primarily using feature point extraction techniques, such as SIFT [18] or SURF [19], and uses the position information of the feature points to estimate the 3D pose of the objects. In addition, a technique that matches the feature points between a 3D CAD model and a 2D object image and uses the PnP algorithm to estimate the 3D pose was introduced [20]. [PERSON] et al. [21] recognized complexly arranged objects of various shapes and colors and segmented the regions of the objects. Then, the rotation transformation was calculated as quaternions to transform the CAD coordinate system to the camera coordinate system for each segmented object. In their paper, they proposed a deep learning network called PoseCNN for obtaining the rotation transformation. They also built the fully connected layer of the network in an end-to-end fashion so that quaternions could be regressed. They used the YCB-Video and LINEMOD datasets to evaluate their proposed technique quantitatively and used both RGB and RGB-D images as the input images. As RGB-D input images were more advantageous for obtaining the rotation information of objects because of the depth and scale information of the objects. Their 6-DoF pose estimation performance was also higher for RGB-D input images. The DenseFusion 6-DoF pose estimation network proposed by [PERSON] et al. [22] used RGB-D images as the input, and RGB and depth features were extracted using a separate CNN network. Then, the features extracted from each image were fused into a global feature using a multilayer perceptron (MLP) to extract features for each image pixel. In the final process, the features of each pixel were used to perform regression on the rotation information of the object with a quaternion model. Their proposed network showed excellent performance and outperformed existing techniques in the ADD-S [18] evaluation. [PERSON] et al. [23; 24] published two deep-learning-based papers related to pose estimation: PVN3D and FFB6D. As both networks use RGB-D images as the input, they have the drawback of having to obtain the depth information in advance using a color camera and, additionally, a depth camera. A common technique used in both studies is to utilize a CNN and an MLP to find keypoints, which are the most important feature points in RGB images and depth images. ResNet34 [9], trained with general ImageNet [25], was used to extract features from RGB images, and PointNet++ [26] was used to extract features from depth images. In addition, keypoints were extracted using an MLP that shares the result of each CNN as the weight. Here, the keypoints have color and depth information. Hence, a 3D transformation matrix that minimizes the error between these keypoints and the keypoints predefined in the CAD model is calculated. In their study, the FFB6D network does not use an MLP and has a characteristic where all the CNN layers that obtain features from the RGB and depth images share the weights. Then, the rotation and translation matrices are obtained by performing depth-wise concatenation and least squares on the shared features extracted by the CNN. To develop a bin picking technology for the automation of logistics, [PERSON] et al.