FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Authors: name: Bowen Wen; name: Wei Yang; name: Jan Kautz; name: Stan Birchfield
Published: 2023-12-13 18:28:09

Abstract: We present FoundationPose, a unified foundation model for 6D object pose
estimation and tracking, supporting both model-based and model-free setups. Our
approach can be instantly applied at test-time to a novel object without
fine-tuning, as long as its CAD model is given, or a small number of reference
images are captured. We bridge the gap between these two setups with a neural
implicit representation that allows for effective novel view synthesis, keeping
the downstream pose estimation modules invariant under the same unified
framework. Strong generalizability is achieved via large-scale synthetic
training, aided by a large language model (LLM), a novel transformer-based
architecture, and contrastive learning formulation. Extensive evaluation on
multiple public datasets involving challenging scenarios and objects indicate
our unified approach outperforms existing methods specialized for each task by
a large margin. In addition, it even achieves comparable results to
instance-level methods despite the reduced assumptions. Project page:
https://nvlabs.github.io/FoundationPose/