# FMT: Unsupervised RGB-D Point Cloud Registration via Fusion Feature Matching with Transformer
## 1 Introduction  
Point cloud registration is a pivotal problem in the domains of computer vision and 3D computer graphics, which are dedicated to achieving precise alignment of point cloud data acquired from heterogeneous sources within a unified coordinate framework. The methodologies related to point cloud registration can be broadly categorized into two main classes: optimization-based methods and feature-based approaches. Optimization-based methods [1, 2, 3, 4, 5, 6] achieve registration by minimizing the error between the source and target point clouds. However, such methods are prone to converging to local optima and exhibit a high dependency on the initial alignment and the quality of the point cloud data. Feature-based methods achieve registration by extracting geometric feature points from the point clouds and matching them accordingly. Representative algorithms include feature descriptor-based approaches such as [7, 8, 9, 10, 11, 12]. These methods demonstrate strong capabilities in representing local geometric information but are notably sensitive to noise and occlusions. In recent years, deep learning has been introduced into the field of point cloud registration [13, 14, 15, 16, 17, 18], utilizing end-to-end frameworks to directly learn registration rules fromraw point cloud data, significantly enhancing the model's ability to represent complex geometric information. Deep learning-based point cloud registration methods typically employ neural networks to extract features from the point clouds, capturing the underlying spatial relationships and structural patterns and thereby achieving more robust alignment. Furthermore, these methods can effectively handle issues such as noise, incompleteness, and partial overlap in point clouds, significantly improving registration accuracy and generalization capability. However, although deep learning-based point cloud registration methods demonstrate excellent theoretical performance, most of these approaches rely on accurate pose or correspondence supervision. In many practical applications, obtaining precise supervisory information is often challenging due to factors such as sensor errors, occlusions, and noise, which severely limits the applicability of these methods in real-world scenarios. For unlabeled datasets, deep learning models typically rely on weights that are pre-trained on other datasets for inference. Nevertheless, substantial distributional discrepancies often arise between datasets, including variations in point cloud density, viewing angles, and other characteristics, which frequently lead to a decline in performance. With the advancement of RGB-D sensor technology, several RGB-D video datasets [19; 20] have been constructed. These datasets seamlessly integrate depth information with rich color channels, offering enhanced semantic representation and significantly advancing the development of unsupervised learning methodologies. UR&R [21] is the first work to leverage unsupervised learning for point cloud registration using RGB-D data, achieving performance competitive with supervised learning methods. It employs differentiable alignment and rendering to enforce photometric and geometric consistency between frames, serving as an effective supervisory signal. Therefore, inspired by UR&R [21], we believe that unsupervised learning methods possess broader application potential and can achieve outstanding performance. Following the publication of UR&R [21], the academic community has witnessed the emergence of numerous outstanding research advancements based on unsupervised learning [22; 23; 24]. However, the aforementioned methods still exhibit certain limitations. In the case of BYOC [22], the extraction of 3D descriptors is performed independently of the visual features, preventing the deep integration between the two modalities. For LLT [23], although geometric information is utilized to assist in visual feature extraction, the potential of RGB-D data has yet to be fully explored.