# Region Pixel Voting Network (RPVNet) for 6D Pose Estimation from Monocular Image
## 2 Materials and Methods
### Proposed Method
#### 2.2.3 Direction Map  
Predicting keypoints with heatmap is widely used in fields such as human posture estimation [20; 21]. In heatmap, the closer a point is to a keypoint, the greater the probability that it is a keypoint. Normal distribution is commonly used to describe this probability. However, this is not a good way to handle cases where keypoint is blocked, since part of heatmap is occluded as well. If the size of the Gaussian kernel is chosen too small, the entire heatmap will not have a significant value. As a countermeasure, the complete heatmap can still be predicted for the occluded keypoint, as if it was not blocked. However, this is actually predicting the keypoint's heatmap with the obstacle's pixels, which have nothing to do with the object. This affects the training performance and results in large prediction errors.  
The proposed method adopts a strategy of direction map [29]. The direction map is defined as follows.  
**Definition 3**.: _For a given Image, \(k\in Image\) and is the target object's keypoint in Image. Then, the direction map of \(k\) in the image is:_  
\[D(k)=\left\{d_{p}=(p,dir)|p\in Image,dir=Indi(p\in obj)\frac{k-p}{\|k-p\|}\right\}\]  
_where \(Indi(p\in obj)=\begin{cases}1&\text{if }\text{ture}\\ NaN&\text{otherwise}\end{cases}\)_  
Figure 4: Driller.  
Figure 5: Duck.  
As introduced in Section 2.2.1, keypoint head consists of eight convolutions with stride 1 followed by one layer of deconvolution with stride 2. If the input feature is a tensor of size \(m_{d}/2\times n_{d}/2\), it outputs a \(m_{d}\times n_{d}\) raw direction map.  
According to Definition 3, it is clear that only pixels belonging to target object contribute to the direction map, which helps to shield the interference caused by the occlusion. Obviously, mask is needed to perform masking operations on raw direction map to get the final direction map. During inference, mask is provided by the mask head introduced in Section 2.2.1; during training, they are provided by ground truth mask.  
The 2D keypoints are inferred from the predicted direction maps through RANSAC voting, as shown in Algorithm 2. Note that Algorithm 2 is designed for only one keypoint, and all keypoints are predicted using the same algorithm.  
```
Input:\(\mathbf{D}(k_{i})\) the direction map for \(k_{i}\) ; \(N_{it}\) Maximum number of iterations ; threshold voting threshold; Output:\(\mathbf{k}_{i}\) the \(i\)th keypoint of target object in image;
1 Initialize:\(t\gets 0\),\(bestscore\gets 0\);
2 while\(t<N_{it}\)do
3 randomly select \(d_{t1},d_{t2}\in\mathbf{D}(k_{i})\) that belongs to target object, calculate intersection \(k_{hypo}\) ;
4 if\(score\big{(}thresh,k_{hypo}\big{)}>bestscore\)then
5\(\mathbf{k}_{i}\gets k_{hypo},bestscore\gets score\Big{(}threshold,k_{hypo} \Big{)}\);
6 end if
7\(t\gets t+1\);
8 if\(1-\Big{(}1-score^{2}(threshold,k_{hypo})\Big{)}^{t}\)then
9 break;
10 end if
11 end for
12 return\(\mathbf{k}_{i}\)
```  
**Algorithm 2**RANSAC voting for a keypoint  
In our model, \(N_{it}\) is set to 512 and \(threshold\) is set to 0.99. The voting score is calculated as follows:  
\[score(threshold,k)=average\Bigg{(}\sum_{d\in\mathbf{D}}{\emph{Indi}\bigg{(}\frac{k-p }{\|k-p\|}\cdot dir>threshold\bigg{)}}\Bigg{)} \tag{4}\]