# Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation
## 3 Proposed Pose Estimation Pipeline
### Estimation of Rotation and Focal Length  
The previous section provided the theoretical background of the 4-DOF-based object pose estimation proposed in this paper. This section describes the method of calculating 4-DoF, that is, 3-DoF for rotation and 1-DoF for the focal length f. Estimating the rotation between the pose of the object detected in the RGB image based on the camera coordinate system and the ShapeNetCore world coordinate system of the matching CAD model has a high complexity of 3-DoF. Previous related studies also used the feature values for a visual change owing to the rotation of the object. For learning-based methods, previous studies have adopted the method of learning these features using a deep learning network. However, as observed in FEB6D, which is a recent learning-based pose estimation method that uses RGB images as the input, even learning all the visual features of the 3-DoF rotation values for an object class requires a large amount of training data and deep CNN layers. Hence, FEB6D is trained on one class and performs pose estimation for that class. In the case of the existing methods using RGB-D images as the input, additional depth information is available. Hence, these methods simultaneously learn the poses of objects from multiple classes and use them. However, in this study, only RGB images are used as the input, and hence, it is difficult to utilize them. In this study, the 3-DoF pose is estimated using only the RGB input images of objects of several classes. Hence, the existing method based on the image features of the objects is not used. Instead, PoseContrast, which is a learning-based method that can estimate the pose  
Figure 4: Definition of the translation ‘t¿’ from the camera to the CAD model coordinates. regardless of the class of the object, is used. PoseContrast is a learning method that uses pose-aware contrast loss to consider only the pose of the object and not its class. The pose contrast loss of the object in the RGB image is based on SimCLR, which applies different types of data augmentation to the image regardless of the class of the object. Then, the feature representations of positive pairs are trained so that the distance between the feature vectors decreases, and the feature representations of negative pairs are trained so that the distance between the feature vectors increases. Based on this principle, PoseContrast learns pairs with different poses as negative pairs and pairs with similar poses as positive pairs. Three datasets--Pascal3D+, ObjectNet3D, and Pix3D--were used to train PoseContrast. The entire dataset comprises 80,696 training images and 42,306 validation images for 121 classes. Each dataset contains Euler angle information (azimuth, elevation, inplane rotation) as the ground truth for learning the poses. The Adam optimizer was used in an end-to-end manner for training, and the batch size was set to 32. The initial learning rate was \(10^{-4}\) at 15 epoch, and the training was conducted for approximately two hours using the V-100 16G GPU. In addition, MoCo v2 (pretrained ResNet-50) was used as the image encoder network. After the rotational pose of the object is estimated, the method for estimating the camera focal length \(t\) of 1-DoF is as follows. To project the CAD model onto the 2D BB image space, the model needs to be expressed as points, and Equations (4)-(8) need to be applied. However, all the CAD models of ShapeNetCore are saved in a mesh format. Therefore, it is difficult to project the vertices of the mesh model onto the image plane and compare the degree of overlap between the projected model and the image information of the actual object. Therefore, in this study, the CAD model expressed in a mesh format was transformed into point clouds, which were then projected onto the BB image. Then, the degree of overlap between the distribution of the point clouds and the segmented region of the object was determined using the IoU. As shown in Figure 5, the CloudCompare [37] software was used to transform the mesh model into point clouds.