# Region Pixel Voting Network (RPVNet) for 6D Pose Estimation from Monocular Image
## 1 Introduction
To find the object's bounding box, [PERSON] et al. [23] used Faster R-CNN [24] and an hourglass architecture to predict the keypoints' heatmaps. To obtain accurate and robust prediction for keypoints, [PERSON] et al. [25] predicted heatmaps from multiple small patches independently and accumulated the results. However, predicting heatmap also has a disadvantage when dealing with occlusion cases, as the occluded keypoints may not produce significant values on the heatmap, which may lead to estimation error. Another kind of two-stage methods produce intermediate values for every pixel or patch and then determine the final result in a manner similar to a Hough voting [26]. For example, [PERSON] et al. [27] used conditional random field to generate the small number of pose hypotheses which will be selected through voting. In the study by [PERSON] et al. [28], the proposed method regresses coordinates and object labels using automatic context and random forest, makes an initial pose estimation through fixed Random Sample Consensus (RANSAC) hypotheses and finally refines the estimation using the uncertainty of coordinates. However, regressing coordinates is hard due to large output space. Thus, PVNet [29], which is the state-of-the-art, predicts vector pointing to a keypoint in the image for every pixel that belongs to the object, and produce the keypoints via a voting procedure, and then a PnP procedure is adopted for final pose estimation. PVNet uses the whole features for prediction. The disadvantage of whole features is that most of them are not relevant to the object to be estimated, considering that the target object generally occupies a very small part of the picture. To estimate the 6D-pose from image with high accuracy in occluded scenarios, a novel two-stage algorithm called RPVNet is proposed. The backbone, object detection network (detection head) and mask prediction network (mask head) are designed based on Mask R-CNN, ResNet18 [30], and Feature Pyramid Network (FPN) [31] is used to construct the backbone; the detection head is built by a simple fully connected network. After inputting the original image, backbone produces features of different scales. With the regions provided by Region Proposal Network (RPN), detection head uses these features to produce the object's bounding box. We point out that predicting an object's keypoints is related to its own pixels and independent of other pixels. Thus, as a main contribution, we use local features instead of the whole features to compute the direction maps. Direction map is a vector field with the direction of each point pointing to its source keypoint. With the region specified by the bounding box, local features are extracted from the whole features through Region of Interest (RoI) Align. Keypoint detection head, which is implemented by a well-designed CNN, uses the local features to generate direction maps. After the direction maps are aligned with original image, the keypoints are recovered by pixel voting using RANSAC. Finally, given the keypoints' prior 3D coordinates, the 6D-pose can be calculated by solving the PnP problem.