Find Any Part in 3D
Authors: name: Ziqi Ma; name: Yisong Yue; name: Georgia Gkioxari
Published: 2024-11-20 18:59:01

Abstract: We study open-world part segmentation in 3D: segmenting any part in any
object based on any text query. Prior methods are limited in object categories
and part vocabularies. Recent advances in AI have demonstrated effective
open-world recognition capabilities in 2D. Inspired by this progress, we
propose an open-world, direct-prediction model for 3D part segmentation that
can be applied zero-shot to any object. Our approach, called Find3D, trains a
general-category point embedding model on large-scale 3D assets from the
internet without any human annotation. It combines a data engine, powered by
foundation models for annotating data, with a contrastive training method. We
achieve strong performance and generalization across multiple datasets, with up
to a 3x improvement in mIoU over the next best method. Our model is 6x to over
300x faster than existing baselines. To encourage research in general-category
open-world 3D part segmentation, we also release a benchmark for general
objects and parts. Project website: https://ziqi-ma.github.io/find3dsite/