# RSCS6D: Keypoint Extraction-Based 6D Pose Estimation
## 1 Introduction  
6D pose estimation is a highly complex problem in computer vision. It involves predicting the position and orientation of 3D objects in space, which requires determining both rotational and translational information for the target object. Typically, this process involves localizing objects using object detection or semantic segmentation before estimating their pose. As a critical concept in computer vision, 6D pose estimation plays a vital role in various applications, including robotic grasping [1, 2], autonomous driving [3, 4], and augmented reality [5]. Traditional methods for pose estimation [1, 6, 7] rely on handcrafted features to establish 2D-3D correspondences between images and 3D object meshes. However, these methods face significant challenges, such as sensor noise, variable lighting conditions, and visual ambiguity. Recent approaches leverage deep learning, using convolutional neural networks (CNNs) or Transformer-based architectures [8] on RGB images to address these challenges. These methods typically provide pixel-level correspondences [9, 10, 11, 12] or predict the 2D image locations of predefined 3D keypoints [13, 14], yielding more robust results. Despite these advances, challenges, such as handling textureless objects and severe occlusions, remain unresolved. To address the challenges of 6D pose estimation and leveraging the decreasing cost of RGB-D sensors, recent studies (e.g., PVN3D [15], FFB6D [16], RCVPose [17], and DFTr [18]) have adopted RGB-D images for 6 DoF object pose estimation. Given the large output space of 6 DoF, existing RGB-D-based methods predict predefined 3D keypoints in space (i.e., establishing sparse 3D-3D correspondences) and then solve for the pose using least-squaresfitting. While these sparse keypoint approaches provide fundamental cues for object pose estimation, they still have several limitations. First, they often involve time-consuming procedures, such as keypoint voting. Second, predicting occluded keypoints remains challenging. Third, they may fail to locate object landmarks under viewpoint changes [19]. Another approach in RGB-D methods aims to directly predict object poses from feature representations generated by neural networks, such as DenseFusion [20] and ES 6D [21]. Uni 6D [22] and Uni 6 Dv2 [23] further integrate RGB and depth feature extraction for direct pose prediction. While these methods bypass the need for explicit 2D-3D correspondence, the pose derivation process is less interpretable, since object poses are regressed directly from feature embeddings. Consequently, object poses estimated using direct pose estimation methods are often less accurate. This weakness has also been observed in related research, such as camera pose estimation (also known as egomotion estimation) [24; 25]. The reasons for the inaccuracy of pose regression methods are elaborated in [26]. To address the limitations of existing RGB-D methods, we propose a novel keypoint cloud extraction method. This method replaces the target point cloud with keypoint cloud to mitigate the impact of redundant information, thereby alleviating the drawbacks of sparse correspondences and pose regression. Our approach involves using dense 2D-3D correspondences, leveraging the RGB channel, and 3D-3D correspondences, utilizing the D channel input to establish correspondences with the 3D object model's point cloud. To better integrate RGB and depth information, we have modified the G2L-Net method [27]. We detect 2D edge information in RGB images using the Sobel edge detection operator and derive point cloud edge keypoints through semantic segmentation masks and 3D-3D correspondences. Additionally, we compute the curvature matrix from the depth map and use a first-order differential Sobel operator combined with masks to supplement point cloud internal keypoint information. The resulting keypoint cloud is fed into an improved pose estimation network, which maintains interpretability and achieves higher accuracy than sparse correspondence methods.