# RSCS6D: Keypoint Extraction-Based 6D Pose Estimation
## 3 Proposed Method
### Keypoint Cloud Extraction
#### 3.3.4 CS Algorithm for Internal Keypoint Extraction  
Curvature is a critical feature for describing the concavity or convexity of a surface, reflecting the degree of variation in the point cloud surface. Points with high curvature values, indicating significant surface changes, are often considered key points in 3D point clouds. In this work, we calculate the curvature of points in a 3D point cloud by leveraging 2D neighborhood relationships from RGB images. The process involves the following steps:  
1. Semantic Segmentation: Using PIDNet algorithm, we obtain the matrix \(\mathbf{C}_{obj}\), which consists of the coordinates of the target object in the 2D image coordinate system: \[\mathbf{C}_{obj}= \begin{bmatrix}x_{1}&x_{2}&\cdots&x_{n}\\ y_{1}&y_{2}&\cdots&y_{n}\end{bmatrix} ,\] (7)
2. Calculate 2D Bounding Box Parameters: From matrix \(\mathbf{C}_{obj}\), we compute the parameters of the 2D bounding box (x, y, w, h) using the following formulas: \[ \begin{array}{c}\mathrm{x}=\mathrm{min}x_{i},i=1,2,\cdots,n,\mathrm{y}= \mathrm{min}y_{i},i=1,2,\cdots,n\\ \mathrm{w}=\mathrm{max}x_{i}-\mathrm{min}x_{i},i=1,2,\cdots,n,\mathrm{h}= \mathrm{max}y_{i}-\mathrm{min}y_{i},i=1,2,\cdots,n,\end{array} \] (8)
3. Generate 2D Coordinate Matrix and Label Vector: Using the bounding box parameters, we generate the 2\(\times\)wh 2D coordinate matrix \(\mathbf{C}_{bbx}\): \[\mathbf{C}_{bbx}= \begin{bmatrix}x_{1}&\cdots&x_{w}&x_{w+1}&\cdots&x_{2w}& \cdots&x_{wh}\\ y_{1}&\cdots&y_{w}&y_{w+1}&\cdots&y_{2w}&\cdots&y_{wh}\end{bmatrix} ,\] (9) Simultaneously, we obtain the corresponding label vector \(\mathbf{L}_{obj}\): \[\mathbf{L}_{obj}= \begin{bmatrix}1_{obj}^{1},\cdots,l_{obj}^{wh}\end{bmatrix} ,l_{obj}^{j}=\left\{ \begin{array}{c}0,\text{ if }f_{mask}\big{(}x_{j},y_{j} \big{)}=0\\ 1,\text{ if }f_{mask}\big{(}x_{j},y_{j}\big{)}=255\end{array} ,j=1,\cdots,wh,\] (10) Here, \(f_{mask}\big{(}x_{j},y_{j}\big{)}\) represents the pixel mask value at coordinate position \(\big{(}x_{j},y_{j}\big{)}\). 4. Calculate 3D Coordinate Matrix: Combining the depth map with the camera model (Equation (2)), we compute the 3D coordinate matrix \(\mathbf{PC}_{bdx}\). \[\mathbf{PC}_{bbx}= \begin{bmatrix}X_{1}&\cdots&X_{w}&X_{w+1}&\cdots&X_{2w}& \cdots&X_{wh}\\ Y_{1}&\cdots&Y_{w}&Y_{w+1}&\cdots&Y_{2w}&\cdots&Y_{wh}\\ Z_{1}&\cdots&Z_{w}&Z_{w+1}&\cdots&Z_{2w}&\cdots&Z_{wh}\end{bmatrix} ,\] (11) For convenience, \(\mathbf{PC}_{bdx}\) can be represented as column vectors: \[\mathbf{PC}_{bdx}= \begin{bmatrix}\mathbf{t}_{1},\mathbf{t}_{2},\cdots, \mathbf{t}_{wh}\end{bmatrix} ,\;\mathbf{t}_{j}= \begin{bmatrix}X_{j}\\ Y_{j}\\ Z_{j}\end{bmatrix} ,\;j=1,\cdots,wh,\] (12)
5. Curvature Calculation: For each point \(M_{j}=\big{(}X_{j},Y_{j},Z_{j}\big{)}\) in \(\mathbf{PC}_{bdx}\), we use Algorithm 1 to compute the curvature \(q_{M_{j}}\). These curvature values form the curvature matrix \(\mathbf{Q}\): \[\mathbf{Q}= \begin{bmatrix}q_{M_{1}}&q_{M_{2}}&\cdots&q_{M_{w}}\\ q_{M_{w+1}}&q_{M_{w+2}}&\cdots&q_{M_{w}}\\ \vdots&\vdots&\ddots&\vdots\\ q_{M_{w(h-1)+1}}&q_{M_{w(h-1)+2}}&\cdots&q_{M_{wh}}\end{bmatrix} ,\] (13)Curvature Gradient Matrix:  
Using the Sobel operator, we process the curvature matrix \(\mathbf{Q}\) to obtain the curvature gradient matrix \(\mathbf{G}_{Q}\):  
\[\mathbf{G}_{Q_{x}}= \begin{bmatrix}-1&0&1\\ -2&0&2\\ -1&0&1\end{bmatrix} *\mathbf{Q},~{}\mathbf{G}_{Q_{y}}= \begin{bmatrix}-1&-2&-1\\ 0&0&0\\ 1&2&1\end{bmatrix} *\mathbf{Q},\mathbf{G}_{Q}=\sqrt{\mathbf{G}_{Q_{x}}^{2}+ \mathbf{G}_{Q_{y}}^{2}}, \tag{14}\]
7. Key Point Selection:  
By setting a specific threshold \(\mathbf{T}\) and combining it with the curvature gradient matrix \(\mathbf{G}_{Q}\), we obtain the curvature label vector \(\mathbf{L}_{Q}\):  
\[\mathbf{L}_{Q}= \begin{bmatrix}l_{Q^{\prime}}^{1},\cdots,l_{Q}^{wh}\end{bmatrix} ,l_{Q}^{j}=\left\{ \begin{array}{ll}0,&\text{if }q_{M_{j}}\leq\mathbf{T}\\ 1,&\text{if }q_{M_{j}}>\mathbf{T}\end{array} ,j=1,\cdots,wh,\right.