. We study a novel problem of semantic labeling of raw 3D models from online shape repositories, composed of highly fine-grained components. Given a 3D shape comprising numerous human-crafted components, our method produces high quality labeling result (indicated by distinct colors). This is achieved by aggregating the components into part hypotheses and characterizing these mid-level elements for robust grouping and labeling of fine-grained components. A majority of stock 3D models in modern shape repositories are assembled with many fine-grained components. The main cause of such data form is the component-wise modeling process widely practiced by human modelers. These modeling components thus inherently reflect some function-based shape decomposition the artist had in mind during modeling. On the other hand, modeling components represent an over-segmentation since a functional part is usually modeled as a multi-component assembly. Based on these observations, we advocate that labeled segmentation of stock 3D models should not overlook the modeling components and propose a learning * Corresponding author: Kai Xu (kevin.kai.xu@gmail.com) Authors' addresses: Xiaogang Wang, Beihang University; Bin Zhou, Beihang University; Haiyue Fang, Beihang University; Xiaowu Chen, Beihang University; Qinping Zhao, Beihang University; Kai Xu, National University of Defense Technology and Princeton University. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2018 Association for Computing Machinery. 0730-0301/2018/11-ART1 $15.00 https://doi.org/10.1145/3272127.3275009 solution to grouping and labeling of the fine-grained components. However, directly characterizing the shape of individual components for the purpose of labeling is unreliable, since they can be arbitrarily tiny and semantically meaningless. We propose to generate part hypotheses from the components based on a hierarchical grouping strategy, and perform labeling on those part groups instead of directly on the components. Part hypotheses are mid-level elements which are more probable to carry semantic information. A multiscale 3D convolutional neural network is trained to extract context-aware features for the hypotheses. To accomplish a labeled segmentation of the whole shape, we formulate higher-order conditional random fields (CRFs) to infer an optimal label assignment for all components. Extensive experiments demonstrate that our method achieves significantly robust labeling results on raw 3D models from public shape repositories. Our work also contributes the first benchmark for component-wise labeling. Fig. 2 . Statistics on component count (left) and size (right) of the models in ShapeNetCore. The left histogram is measured only for car models. In the right one, size is measured by the ratio of bounding box volume between a component and the whole shape.
Wang et al. REF collected 1,016 3D models from 10 object categories and trained neural networks for grouping and labeling fine-grained part components.
Learning to Group and Label Fine-Grained Shape Components
ACM Transactions on Graphics, 2018
Mathematics Computer Science