# Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation
## 4 Experiments
### Comparison with a Triplet Loss Learning Method  
The performance of the proposed method is also compared with a learning based CAD model retrieval method [30]. In the deep learning architecture in [30], a triplet loss learning network is employed for CAD model retrieval. This method also retrieves the top-n matching CAD rendering images for candidate models, and a key point matching is used to determine the final best matching model. We also employ the triplet loss learning network for comparison with our method as shown in Table 4.  
The triplet loss learning network in [30] composed of two VGG encoders to train the positive and negative features between the RGB and CAD rendering images in a feature embedding space. Here, the features from the RGB images become the anchor in the feature space. For each RGB input image, we use one matching CAD image as the positive label and ten mis-matching CAD images as the negative labels. The mis-matching CAD images are randomly selected from all categories. As the encoder, the VGG16 model is used and it was trained with ImageNet2014. For training, total 592 RGB images are used and the batch size is set to 6. As the optimizer, SGD is used and the learning rate is set to \(2\times 10^{-6}\). To test the triplet loss network, total 100 RGB images are used which have not been included in the learning. Furthermore, to improve the performance of the triplet loss network, the negative mining strategy is also used. For negative mining, the CAD images of the bottom-7 negative images are reused for training the network.  
Figure 10 shows the final image alignment results using the proposed 4-DoF pose estimation. Onto the real image space, the matched CAD model, which is the top-first CAD model from the similarity measure, is projected and overlapped. To project the matched CAD model, we convert the mesh structure of the CAD model to point clouds using CloudCompare. This alignment results show that the pose of the matched CAD model is reasonably accurate for the application to the extended reality. As shown in the figure, there is still overlap errors in some objects, which is caused by shape mismatch between the real object and the CAD model. For example, the result of _Bag_ object show that the matched CAD model couldn't perfectly overlap with the real object area. This is due to the fact that there is no perfectly matching shape among our CAD candidates.  
We present Figure 10 only for showing examples of CAD model alignment in the input images, not showing the performance measurement of the proposed pose estimation. For quantitative measurement of the alignment performance, the shape of the CAD model should be same with the real object in the image so that the alignment error is measured in  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **1-st Rank** & **2-nd Rank** & **3-rd Rank** & **Top-3** \\ \hline Triplet loss learning [30] & 31 & 18 & 19 & 68 \\ (w/o negative mining) & 29 & 25 & 17 & 71 \\ Triplet loss learning [30] & 56 & 13 & 4 & 73 \\ (w/negative mining) & 56 & 16 & 6 & **78** \\ Proposed (SM1) & 51 & 15 & 12 & **78** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of CAD retrieval performance with triplet loss network [30]. (100 untrained RGB images are teste.d).  
the image space. However, this paper uses the Internet database for the most-similar CAD retrieval, thus the quantitative alignment error analysis is not addressed.