Novel Object 6D Pose Estimation with a Single Reference View
Authors: name: Jian Liu; name: Wei Sun; name: Kai Zeng; name: Jin Zheng; name: Hui Yang; name: Lin Wang; name: Hossein Rahmani; name: Ajmal Mian
Published: 2025-03-07 17:00:41

Abstract: Existing novel object 6D pose estimation methods typically rely on CAD models
or dense reference views, which are both difficult to acquire. Using only a
single reference view is more scalable, but challenging due to large pose
discrepancies and limited geometric and spatial information. To address these
issues, we propose a Single-Reference-based novel object 6D (SinRef-6D) pose
estimation method. Our key idea is to iteratively establish point-wise
alignment in the camera coordinate system based on state space models (SSMs).
Specifically, iterative camera-space point-wise alignment can effectively
handle large pose discrepancies, while our proposed RGB and Points SSMs can
capture long-range dependencies and spatial information from a single view,
offering linear complexity and superior spatial modeling capability. Once
pre-trained on synthetic data, SinRef-6D can estimate the 6D pose of a novel
object using only a single reference view, without requiring retraining or a
CAD model. Extensive experiments on six popular datasets and real-world robotic
scenes demonstrate that we achieve on-par performance with CAD-based and dense
reference view-based methods, despite operating in the more challenging single
reference setting. Code will be released at
https://github.com/CNJianLiu/SinRef-6D.