Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objects
Authors: Yuki Kawana
YUSUKE Mukuta
Tatsuya Harada
Venue: ICLR 2022 Submitted (2022)
Keywords: unsupervised part decomposition
shape abstraction
3D shape representations
generative models
computer vision

Abstract: Articulated objects exist widely in the real world. However, previous 3D generative methods for unsupervised part decomposition are unsuitable for such objects, because they assume a spatially fixed part location, resulting in inconsistent part parsing. In this paper, we propose PPD (unsupervised Pose-aware Part Decomposition) to address a novel setting that explicitly targets man-made articulated objects with mechanical joints, considering the part poses. We show that category-common prior learning for both part shapes and poses facilitates the unsupervised learning of (1) part decomposition with non-primitive-based implicit representation, and (2) part pose as joint parameters under single-frame shape supervision. We evaluate our method on synthetic and real datasets, and we show that it outperforms previous works in consistent part parsing of the articulated objects based on comparable part pose estimation performance to the supervised baseline.

Under review as a conference paper at ICLR 2022
UNSUPERVISED POSE-AWARE PART DECOMPOSITION FOR 3D ARTICULATED OBJECTS
Anonymous authors Paper under double-blind review
ABSTRACT
Articulated objects exist widely in the real world. However, previous 3D genera- tive methods for unsupervised part decomposition are unsuitable for such objects, because they assume a spatially fixed part location, resulting in inconsistent part parsing. In this paper, we propose PPD (unsupervised Pose-aware Part Decom- position) to address a novel setting that explicitly targets man-made articulated objects with mechanical joints, considering the part poses. We show that category- common prior learning for both part shapes and poses facilitates the unsupervised learning of (1) part decomposition with non- cetn--imiiiiebaseeeeeee implicit representa- tion, and (2) part pose as joint parameters under single-frame shape supervision. We evaluate our method on synthetic and real datasets, and we show that it out- performs previous works in consistent part parsing of the articulated objects based on comparable part pose estimation performance to the supervised baseline.
INTRODUCTION
Humans capable of recognizing complex shapes by decomposing them into simpler semantic parts, Researchers have shown that infants learn to group objects into semantic parts using the location, shape, and kinematics
cue (Spelke et al. 1995 Slater et al.) 1985 Xu & Carey (996).
Moreover, even very young infants can learn to reason about kinematics using non sequential single frames (Shirai Imura (2014] Kourtzi Kanwisher 2000). Although humans can naturally achieve such reasoning, it is challenging for machines, particularly in the absence of a rich supervision.
Generative part decomposition and abstraction methods have a long-standing history in computer vision (Roberts) 1963 Binford 1971). Learning to represent complex target shapes with simpler part components has wide range of applications, such as structure modeling (Mo al., 2020
Roberts al. 202 1) and unsupervised 3D part parsing (Chen et al. 2020 Paschalidou et al. 2021
Tulsiani et al. 2017). Previous studies have mainly focused on non-articulated objects. Because they exploit the consistent part location cue group shapes into semantic parts, these approaches are unsuitable for decomposing articulated objects when considering the kinematics of dynamic part locations. In contrast, there exist discriminative approaches targeting man-made articulated objects for part segmentation, in addition to part pose estimation from single-frame input. However, they require explicit supervision, such as segmentation labels and joint parameters (Yi et al. [018 Xiang et al. 2020; Li et al. 2020). Removing the need for such expensive supervision has been an important step toward more human-like representation learning (Becker & Hinton 1992).
In this study, as novel problem setting, we investigate the generative part decomposition task for man- articulated objects with mechanical joints, considering part poses as part kinematics, in an unsupervised fashion. Specifically, we consider the revolute and prismatic parts with a of-redom- joint state as the part kinematics because they cover most of the kinematic types that common man-made articulated objects have (Xiang et al. 2020 AAbbatematteo et al.) 2020) Michel al. 2015). This task aims to learn consistent part parsing as a generative shape abstraction approach similar to (Chen et al., 2019b) for man-made articulated objects with various part poses from single- frame shape observation. An overview is shown in Figure This task expands the target of the current generative part decomposition' applications articulated objects in novel ways, such as part pose consistent part segmentation and part pose estimation. To realize the task, we identify the two challenges; (1) for pose-aware part decomposition, the model must consider the kinematics

---

Under review as a conference paper at ICLR 2022
Johnt rectior
Part WiS
pivot point
GT
BAE-NO
implish halld
Joint stand
Curs
Impu: Point cloud
Oulput: part shapee and posss
BSP
NSD
Figure I: (Left) Even through independent observations, infants can build a mental model of the articulated object for part parsing based on its kinematics. (Middle) Likewise, we propose an unsu- pervised generative method that learns to parse the single-frame, unstructured 3D data of articulated objects and predict the part-wise implicit fields as well as their part poses as joint parameters. (Right) Our approach outperforms the previous works in consistent part parsing for articulated objects.
Part
Part pose estimation
Generative
Unsupervised
segmentation
ANSCH
[Li ct al.
2020
NASA
Deng el al. 2020b
Nucral Parts
Paschalidou
2021
Ours
Table 1: Overview of the previous works. We regard a method as unsupervised if the checked tasks can be learned only via shape supervision during training.
between possibly distant shapes to group them as a single part and (2) has to disentangle the part poses from shape supervision. A comparison with previous studies is presented in Table
To address these challenges, we propose PPD (unsupervised Pose-awar Part Decomposition) that takes an unsegmented, single-frame frame point cloud with various underlying part poses as an input. PPD reconstructs part-wise shapes transformed using the estimated joint parameters as the part poses. We train PPD as an autoencoder using single-frame shape supervision. PPD employs category-common decoders to capture category-specific rest-posed part shapes and joint parameters, Learning to trans- form the rest-posed shapes properly disentangles shape and pose, and (2) restricting the position of the parts by the joint parameters forces shapes in distant space that share the same kinematics to be recovered as the same part. We also propose series of losses, including an adversarial loss, to regu- larize the learning process. Furthermore, we employ non- non-primitee con--pmmmiiieebbrn based part shape representation and utilize deformation by part poses to induce part decomposition, in contrast to previous works that employ primitive shapes and rely on its limited expressive power as an inductive bias.
Our contributions are summarized as follows: (1) We propose a novel unsupervised generative part decomposition method for man- mane articulated objects based on part kinematics. (2) We show that the proposed method learns non-primitive-bitt based implicit field the decomposed part shapes and the joint parameters as the part poses, using single-frame shape supervision. (3) We also demonstrate that the proposed method outperforms previous generative part decomposition methods in terms of semantic capability (parsimonious shape representation, consitent part parsing and interpretability of recovered parts) and show comparable part pose estimation performance to the supervised baseline.
2 RELATED WORKS
Existing unsupervised generative part decomposition studies mostly assume non-articulated objects in which the part shapes are in fixed 3D location (Tulsiani et 2017; Paschalidou et al. 2020 Chen et al. 2019b; 2020: Deng et al. 2020a| Kawana et al. 2020)) or also targeting human body and hand shapes without considering part pose (Paschalidou et al. 202T. They induce part decom- position by limiting the expressive power of the shape decoders by employing learnable primitive shapes. Closest work of ours is BAE-Net (Chen al.) 2019b). whose main focus consistent part parsing by generative shape abstraction. It also employs a non-primitive-basse implicit field as the part shape representation, similar to ours. However, it still limits the expressive power of the shape decoder using MLP with only three layers. In contrast, with the consistent kinematics and induces part decomposition through rigid transformation of the reconstructed part shapes with the estimated part poses to make the decomposition pose-aware.
approach assumes parts to be dynamic
A growing number of studies have tackled the reconstruction of vategory-specifer natural articulated objects with a particular kinematic structure, such the human body and animals. Representative works rely on the use of category-specins template models as the shape and pose prior (Loper et al. 2015 Zuffi et al. 2017 Bogo al. 2016 Zuffi et al., 2019. Kulkarni et al. 2020). Another body
2

---

Under review as a conference paper at ICLR 2022
Shape vector
Part poses
Coordinate
potc
tranefomation
P
Implist field
[D]
[0] x] 2-1
[0]] f
X
max 0
Input: Point cloud
Output Impliii field
Fncodar
Posa
latont vectors
Pose decoders
Shape decoders
Figure 2: Model overview. To infer implicit field Ô based on part poses {Bi and part-wise implicit fields {O:),= the aategory-common decoders FP and (FA" Ji=1 capture pose biases and shape priors, the instance-depen deeendded decoders F and {FAN Ti=1 infer target specific components.
of works reconstruct target shapes without templates, such as by reconstructing a part-wise implicit field given part pose as an imput (Deng et al. 2020b) or focusing on non-rigid tracking of the seen samples (Božič et al., (2021) In contrast, our approach focuses on man man-made articulated objects with various kinematic structures. Our approach learns the shape and pose prior during training, without any part pose information either as supervision or input, and is applicable to unseen samples.
In discriminative approaches, a number of studies have focused on the inference of the part seg- mentation of the input point cloud and part poses as joint parameters (Li et al.) 2020; Xiang et al.) 2020 Abbatematteo al. 2020) targeting man-made articulated objects. These approaches require expensive annotations, such as part labels and ground-truth joint parameters. Moreover, they require category-specific prior knowledge of the kinematic structure. contrast, our model is based on gen- erative approach and is category agnostic. Moreover, it only requires shape supervision during train- ing. A very recent work (Huang et al. 2021 assumes an unsupervised setting where multi-frame, complete shape point clouds available for both input and supervision signals during training and inference. Whereas our approach assumes a single-frame input and shape supervision, it also works with partial shape input during inference. Note that, in this study, the purpose of part pose estima- tion is, as an auxiliary task, to facilitate consistent part parsing. is not our focus to outperform the state-of-te teef-the-rtt supervised approaches in part pose estimation.
3 METHODS
In our approach, the goal to represent an articulated object as set of semantically consistent part shapes based on their underlying part kinematics. We represent the target object shape as an implicit field that can be evaluated at an arbitrary point x E RR3 3D space as O R. - [o, 1], where {x E R O(x) = 0} defines the outside of the object, {x E R3 O(x) 1} the inside, and (x E R (xx) 0.5) the surface. Given point cloud I E P x3 as an input, we approximate the object shape using a composite implicit field Ô that is decomposed into a collection N parts, where Pa is a number of points in the point cloud. The i-th part has an implicit field Ô; R x R + [0, 1] as part shape and part pose B; SE(3). We ensure that O is approximated as O(x) Rs Óx I, (B.) through the losses.
An overview of PPD is shown in Figure 2 PPD employs autoencoder architecture, and is trained under single category setting. Given point cloud I, the encoder derives the disentangled shape latent vector z" and the pose latent vectors z" and zP-s, Category- tures joint parameter biases given p Instance-dependent pose decoder EPP. models residual joint parameters to the biases given z?. The part-wise category-common shape decoder E captures category-commont shape prior. Given z and conditioned by F.r... instance-dependent shape de- coder F infers residual shape details of the target shape to decode part-wise implicit field Ó.- We discuss the details about FP.* and Frie in Section 3.1 and p. and F.s. Section 3.2
pose decoder FPEe cap-
3.1 PART POSE REPRESENTATION
We characterize part pose B; by its part kinematic type y/ E {fixed, prismatic, revolute) and joint parameters. Each y is manually set as hyperparameter. The joint parameters consist of the joint direction u; E R with the unit norm and joint state 8; E R+ Additionally, the "revolute" part has the pivot point qi E R We refer to the joint direction and pivot point as the joint configuration. For the "fixed" part, set B, identity matrix because no transformation is applied. For the
3

---

Under review as a conference paper at ICLR 2022
"prismatic" part, we define B; Ts,, where T(-) represents homogeneous translation matrix given the translation in IR2, and 8; and u, represent the translation amount and direction, respectively. For the "revolute" part, set B; matrix given the rotation representation, and &; and u; represent the axis-angle rotation around the axis i1 by angle 8;- In human shape reconstruction methods using template shape, its pose is initialized to be close to the real distribution to avoid the local minima (Kanazawa et al. [0118) Kulkarni et al. 2020). Inspired by these approaches, we parametrize the joint direction as [u;; [;; 1/, where c; is constant directional vector with the unit norm working as the initial joint direction as a hyperparameter and r, E IS represents the Euler-angle representation working as a residual from the initial joint direction e,, This allows us to manually initialize the joint direction in a realistic distribution through e; by initializing r, = 0. Figure 3
)(s,, uz), where R(-) denotes a homogeneous rotation
illustrates the joint parameters.
Based on our observations, we assume that the joint con- figuration has category-common bias, while the joint state strongly depends on each instance. This is because the location of each part and the entire shape of an object can constrain the possible trajectory of the parts, which is defined by the joint configuration. To illustrate this idea, propose to decompose the joint configuration into category-common bias term and an instance-dependent residual term denoted as r; rf + rr and q, q + . respectively. We employ the category-commen } and {qğ i A") A"). where AF {i [N] y + fixed), A {i E [N] yı revolute). }. 2 denotes a pose latent vector, and qt(.) is latent vector quantization operator following VQ-VAE (Razavi al. 2019). The operator qt(-) outputs the nearest constant vector c" to the input latent vec- tor 2h among the Nºp candidates. Instead of using single constant vector, the model selects con- stant vector among multiple constant vectors to capture the discrete, multi-modal category-comme biases. We also employ instance-dependent pose decoder (zP) that outputs {s; i A?), {rī l E }.). and {qī i E J... We constrain the possible distribution of the joint configuration around the category-common bias by the loss function explained in Section 3.3 This constraint in- centivizes the model to reconstruct the instance-dependent shape variation by the joint state, which constrains the part location along the joint direction. This kinematic constraint biases the model to represent the shapes having the same kinematics with the same part. Because the previous works (Kawana et al. 2020 Deng et al. 2020a Paschalidou et al. 2019) do not impose such constraint on the part localization, learned part decomposition is not necessarily consistent under different poses.
Direction
Direction Residual maadmidres direction si
Joint
Residual
Joint direction My i-th part
Initial
Initial direction
diraction
Pivol point
Rotation amounl
5-th part
Trarslation Sy
Revolute
Prismatic part
Figure 3: Geometric relationship be-
tween the joint parameters.
pose decoder (qt"(q )) which outputs {rş l
3.2 PART SHAPE REPRESENTATION
We propose a non-primitive- part shape representation that is decomposed into the category- common shape prior and instance-dependent shape details. We employ MLP-based decoders to model a part-wise implicit field. We capture the category-common shape prior using the category- common shape decoder F.t.. R + R. Because F does not take latent vector from the encoder, it learns an input-indeppe indepedddd,,,,tt, rest-posed part shape template as the category-common shape prior. We also employ an instance deteede-de shape decoder p. RR3 x RRd - R to capture the additional instance-dependen shape details conditioned with the shape prior, where d is the dimension of the shape latent vector z². Given E:ee and F. we formulate a part-wise implicit field Ō as follows:
Ó, o(F (x, (x((x))
where (-) represents the sigmoid function and 0;(x) = (F. (x)). For brevity, we omit 7 in Ó and simply denote it as (x).. Given the part poses [ as wast locally rigid deformation, we formulate Ô as the composition of (Ó.,.. defined as Ō(x 1, B)= = max, (O, As in the piecewise rigid model of (Deng et al. 2020b), coordinate transformation B, x realizes locally rigid deformation by B, of the part-wise implicit field by querying the rest-posed indicator. Note that, although we set the maximum number of parts N, the actual number of parts used for recon- struction can change; it is possible that some parts do not contribute to the reconstruction because of the mar operation or simply because Ó < 0.5 for all 3D locations. In Equation [ we experi- mentally found that conditioning F by Ó through multiplication rather than addition effectively prevents p from deviating largely from F.c.c. This regularization induces the unsuperivsed part
4

---

Under review as a conference paper at ICLR 2022
decomposition. Considering reconstructing the target shape by single i-th part, since the multiplica- tion makes it difficult to output shapes that deviating largely from the category-common prior shape, the large shape variations of target shapes expressed by B; regarded the global pose of the reconstructed shape. However, the datasets' large shape variations in target shapes are due to the various local poses of multiple part shapes. Therefore, the large shape variations of target shapes cannot be expressed only by the single part and its part pose B, Thus, as an inductive bias of the unsupervised part decomposition, the model is incentivized to use a composition of multiple parts to express the shape variations due to various local part poses. In the learning process, the model first tries to reconstruct the target shapes with single part: then with multiple parts. Lastly, it starts to deform each part to express the shape variations. During the learning process, the part poses are disentangled from the shape supervision transform the part shapes in a way that minimizes the reconstruction loss. For the visualization of the learning process of part decomposition, see Figure 12 in the Appendix.
3.3 TRAINING LOSSES
Shape losses. To learn the shape decoders, we minimize the reconstruction loss using the standard binary cross-entropy loss (BCE) defined as:
Lyee Avec ACe
where (x B) = max; (B, }. and Arec and Afec are the loss weights. The second term in Equation 2 is essential for stable training; it facilitates fast learning of (PNN") so that {F N can be correctly conditioned in the early stage of the training process. Moreover, because the locally rigid deformation of the shape, the volumes of the shape before and after the deformation should not be changed by the intersection of parts; we formulate this constraint follows:
consider
Loot
Avot (Ex [relu(max F. (B," x, z²)}] Ex [relu(max (x,
(x,z"))))²
(3)
Joint parameter losses. For the joint parameters qi and r; prevent instance-dependent term from deviating too much from the bias term, we regularize them by the loss:
Lddy Adev
E Nr
Σ |ril
(4)
No
SEA"
TEAP
where N" |A"), N AA), and Adeu is the loss weight. Moreover, we propose a novel regu- larization loss that constrains the pivot point with the implicit fields. We assume that the line in 3D space, which consists of the pivot point and joint direction, passes through the reconstructed shape. The joint should connect at least two parts, which means that the joint direction anchored by the pivot point passes through at least two reconstructed parts. We realize this condition as follows:
Lioc
Aine Nr
Σ
min |/q: - x|| min// - x|| + min ||q: x1) xxss
(5)
xE
2
xE:,,,
YEAr
where Sat (x R3 0x()) 1}. S. fx ES O xx) 0.5). S.
{x
R3 ÓB 'x) > 0.3, j E A" (1), and Vioc is the loss weight. Note that Lioe is self-regularizing and not supervised by the ground-truth part segmentation. See Figure [3] in the Appendix for an illustration of Lioc- To reflect the diverse part poses, we prevent the joint state 8; from degenerat- ing into static state. In addition, prevent the degeneration of multiple decomposed parts from representing the same revolute part, we encourage the pivot points to be spread. We realize these requirements by the loss defined as:
Lear
1
Σ YEAP
Avar: stdsy (s.)
Avar,
Σ cxp JEAT
|
( V
(6)
NP
where stdæ (-) denotes the batch statistics of the standard deviation, is a constant that controls the distance between pivot points, and Aver, and Avar, are the loss weights. Lastly, following the loss proposed in (Razavi et al. 2019), the pose latent vector zic is optimized by the loss:
Log || sg(c")]l
where sg denotes an operator stopping gradient the backpropagation.
5

---

Under review as a conference paper at ICLR 2022
Adversarial losses. Inspired by human shape reconstruction studies (Chen et al. 2019a Pavllo
ct al. 2019), we employ the adversarial losses from WGAN- (Gulrajani et al. 2017 to regularize the shape and pose in the realistic distribution. The losses are defined as:
Ladua = Xadua (LE [D()) - Em-F, [I())) + EanPa [(]V&D(0)] - -
(8)
Lados = rado, ( (-E (D))]]
where D(-) is a discriminator; z is a sample from the reconstructed shapes Pg transformed by the estimated joint configuration and randomly sampled joint state maximum motion amount h2 treated as a hyperparameter; æ: is a sample from the ground-truth shapes Pr; is a sample from F2. which is set of randomly and linearly interpolated samples between T and æ; and Aade, and Aadva are the loss weights. As an imput to D, we concatenate the implicit field and corresponding 3D points to create a 4D point cloud, following (Kleineberg et al. 2020).
Uniform hz). with the
3.4 IMPLEMENTATION DETAILS
We use the Adam solvers Kingma& & Ba, 2014) with a learning rate of 0.0001 to optimize the losses: Liotal, Lree + Lyol + Log + Ldev + Lioe + Lear + Ludu, (sum of Equations 2 and Ltotala = Ladva (Equation 8). with a batch size of 18. For the input, we use the complete shape point cloud with 4096 points sampled from the surface of the target shape, unless otherwise noted. For the ground-truth implicit field, we use 4096 coordinate points and their corresponding indicator values. We set the loss weights as follows: Aree = 0.01, Xfe = 0.001, Adey = 0.1, Avar, = 0.1, Atoe = 100, Avar, = 0.01, Avol 1000, Aude, 0.65, and Audez 0.35. We set v = 0.01 in Lear and N₂. = 4 for qt(.). For h2 in Laden, we set to z respectively. Note that we experimentally found that it does not constrain the model predict s; larger than h; to reconstruct the target shape. Because we do not impose any geometric constraints on the part shapes, we set the number of parts for each part kinematics yi as its maximum number in the datasets plus an additional one part for over-parameterization.. The detail of the datasets is explained in Section 4 We set N 8, which consists of one "fixed" part, three "revolute" parts, and four "prismatic" parts. For the initial joint direction e;, for each "revolute" part, we set it to the (+2, -2 +V directions, and for each "prismatic" part, we set to the + direction. We use the same hyperparameter for all categories, without assuming the category-specing knowledge. We train our network in two stages following (Chen et al. 2020): first, we train it on an implicit field of 16 grids and then on 323 grids. During the training, the max operation is substituted with LogSunExp for gradient propagation to each shape decoder. See Appendix for further training details.
6 and
2 and 0.4 the "revolute" and "prismatic" parts,
Network architecture. We use the PointNet (Qi et al. 2017)-based based architecture from (Mescheder al. 2019) an encoder E and the one from (Shu et al. 2019) as a discriminator D. Our shape de- coders [F C Ji=1 and {F,"* ) are MLP with sine activation (Sitzmann et al. (2200) for uniform activation magnitude suitable for propagating gradients to each shape decoder. For the category- common pose decoder Fr.e, we use two separate networks of MLP, namely, FF and Fp.e. For the instance-dependent pose decoder FP.s, we employ MLP with a single backbone having multiple output branches. See Appendix A for further architectural details.
4 EXPERIMENTS
Following the recent articulated pose estimation study (Li et al. 2020), we evaluate our method on five categories with various joint configurations from two synthetic datasets: Motion dataset WWang et al. (2019) for the oven, eyeglasses, laptop, and washing machine categories, and SAPIEN dataset (Xiang et al.) 2020) for the drawer category. Each category has a fixed number of parts with the same kinematic structure. We generate 100 instances with different poses per sample, generating 24k instances in total. We divide the samples into the training and test sets with a ratio of approximately 8:2. We also normalize the side length of samples to 1, following MMsccheerr et al. 2019). For further details of the data generation, see Appendix C To verify the transferability of approach trained from RBO dataset (Martin- Marttin-atinnn et al. 2018) and Articulated Object Dataset (Michel et al. 2015). which is the intersecting category with the synthetic dataset.
Datasets.
synthetic data to real data, use the laptop category

---

Under review as a conference paper at ICLR 2022
Eye-
Washing machine
of
Drawer
glasses
Oven
Laptop
mean
parts
BAE
CChen et al.
2019b
6.25*
11.11*
73.06
25.11*
80.30
39.17
1.42/8
BSP
CChnn ct al.
2020
66.31
70.69
81.65
76.68
87.92
76.65
27.50/256
NSD
Kawana el al. 2020
38.39
42.11
74.67
74.44
89.11
63.75
10
NP
(Paschalidou et al. 2021
60.57
64.69
85.41
86.23
74.65
74.31
5
Ours
74.73
66.18
82.07
86.81
95.15
80,99
4.16/8
Table Part segmentation performance in label IoU. Higher better. The starred numbers indicate the failure of part decomposition and that only one recovered part represents the entire shape. The average and the predefined maximum numbers of recovered parts or primitives are shown before and after the slash, in the last column. Ours uses more than one part average with the least number.
Baselines. We compare our method with the state-of-the-art unsupervised generative part decom- position methods with various characteristics: BAE-Net (Chen al. 2020) (non-primitiie- part shape representation), BSP-Net (Chen et al. 2020) (primitive- part shape representa- tion with part localization by 3D space partitioning). NSD (Kawana et al. 2020 and Neural Parts (Paschalidou et al. 202T) denoted as NP (primitive-based part shape representation with part local- ization in I )) For BSP-Net, we train up to 323 grids of the implicit field instead of 64 grids in the original implementation to match those used by other methods. For NSD and Neural Parts, we replace its image encoder with the same PointNet-based encoder in our approach. For the part pose estimation, we use NPCS (Li et al., 2020) as the baseline. NPCS performs part-based registration by iterative rigid-body transformation, which is a common practice in articulated pose estimation of rigid objects. Note that NPCS assumes that part segmentation supervision available during training and part kinematic type per part is known, which we do not assume in both cases. See Appendix B. for further training details of the baselines.
Metrics. For the quantitative evaluation of the consistent part parsing as a part segmentation task, we use the standard label IoU, following the previous studies (Chen et al.) 2019b; 20200 Deng (et al., 2020al Kawana et al. 2020). As our method is unsupervised, we follow the standard initial part labeling procedure using training set to assign each part a ground-trute truth label for evaluation purposes following (Deng et al., 2020a Kawana et al., 2020). A detailed step can be found in Appendix D.I For the part pose evaluation,
evaluate the 3D motion flow of the deformation from the canonical pose to the predicted pose measured as the endpoint error (EPE) (Yan & Xiang, 2016 which is commonly used metric for pose estimation of articulated objects (Wang et al.,
2019 Božič et al. 2021). We scale it by 102 in experiment results. Finally, we report F-score, Chamfer LI distance and volumetric IoU the shape reconstruction accuracy metrics evaluated on the meshified implicit field sampled on 323 grids using marching cubes (Lorensen & Cline 1987). We use 100k points for the above three metrics, following (Mescheder et al., 2019 Deng et al.,
2020a).
4.1 SEMANTIC CAPABILITY
We evaluate the semantic capability of our approach in part parsing. As part decomposition ap- proaches aim to learn 3D structure reasoning with small a number of ground-truth labels as possible, it is preferable to obtain the initial manual annotations with as few numbers of shapes as possible. This requirement is essential for articulated objects, which have diverse shape variations owing to the different articulations. As our approach is part pose consistent, we only need a minimal variety of instances for the initial manual labeling. To verify this, we evaluate the part segmentation performance using only the canonically posed (joint states were all zero) samples in the training set. See Appendix (..e for further studies on pose variation for the initial annotation. The evaluation results are shown in Table 2 We also show the number of parts or primitives that each model uses in the last column of the table. Our approach outperforms all the previous works on average. The segmentation results are shown in Figure 4 The same color indicates the same segmentation part. The grey color of the part segmentation results of BSP-Net indicates that a primitive not labeled in the initial annotation is chosen to segment the surface area. Visualization procedure can be found in Appendix D.2 Our model uses a much smaller number of parts than BSP-Net (Chen et al.) 2020); however, it still performs the best. This shows that model is more parsimonious, and each part has more semantic meaning in part parsing. For additional visualization of our part segmentation result, Appendix E. We also visualize the generated part shapes in Figure 5 We can that
7

---

Under review as a conference paper at ICLR 2022
No zal Parts
Ours
Net
Figure 5: Visualization of parts and primitives. The boxes represent the parts primitives used to reconstruct the semantic parts.
GT
BAE
BSP
NSD
NP
Ours
Figure 4: Visualization of the part segmentation. Mesh re- construction is shown inside box. Best viewed in zoom.
8SP.NET
Eye-
Washing machine
Drawer
glasses
Oven Laptop
mean
NPCS [Li ul, 2020
Oure (shape)
1.598
1.087
2.702
0.751
1.594
1.546
(Supervised)
Ours
Ours (oint state)
3.452
2.631
3.360
2.546
2.529
2,903
(Unsupervised)
Source
Interpolation
Target
Table 3: Part pose estimation performance in EPE. Lower is better.
Figure 6: Interpolation in terms of shape and joint state.
one dynamic part is successfully reconstructed by single implicit field. This demonstrates the ad- vantage of using non-primitive based part shape representation: complicated grouping mechanism of primitive shapes based on part kinematics is not needed. Also, our part shapes are more semantic and interpretable than the previous works. Moreover, our part shape representation exhibits the part shape with disconnected shapes, which the previous single primitive shape cannot express.
Disentanglement between the part shapes and poses. Because our approach disentangles shape supervision into part shapes and poses, realizes pose-aware part decomposition. To verify the learned disentanglement, we visualize the interpolation results of part shapes and joint states as part poses in Figure 6 In the middle row, we show the shape interpolation between the souree and the target while fixing the joint state of the source to maintain the same part pose. The shape is smoothly deformed from the source to the target maintaining the original pose. In the bottom row, we interpolate the joint state between the source and the target; the joint state changes from the source to the target maintaining the shape identity of the source shape. Our model successfully disentangles the part shapes and poses, unlike previous methods as shown in the top row.
4.2 PART POSE ESTIMATION
To validate whether the predicted part decomposition is based on the reasonable part pose estima- tion, we quantitatively evaluate the performance. Because we train our model without specifying a canonically posed shape, we use the estimated deformation between the target instance and the canonically posed instance of the same sample as the estimated part pose to align with the predic- tion of the supervised baseline. We present the evaluation results in Table 3 We show the supervised rigid registration approach NPCS (Li et al.) 2020) only as a reference. Our method is comparable with NPCS, with the same order of performance. Note again that are not attempting to outper- form supervised pose estimation methods; rather, we aim to show that our unsupervised approach decompose parts based on reasonable part pose estimation. See Appendix F for further results.
4.3 RECONSTRUCTION
We quantitatively validate whether PPD learns a reasonable shape representation rather than de- generated to ignore a target shape. We use BAE-Net (Chen et al., (019b) as our baseline because it also focuses on part parsing through generative shape abstraction, and it also employs the same non-primitive part shape representation. The results are presented in Table This study focuses on part parsing based the challenging part pose estimation, thus persuing the state-of-the- -ftt---rr-- shape reconstruction accuracy as in (Chen et al. 20200) Kawana et al.) 2020; Paschalidou et al.] 202T is out of focus. We show their results only reference. We can see that PPD outperforms BAE-Net.
8

---

Under review as a conference paper at ICLR 2022
F-score 1
CD1
IoUt
# of params.
# of parts.
BSP
(Chen et al.
2020
51.85
1.471
55.02
443.3
27.50/256
NSD
Kawana al.
2020
52.59
1.766
51.32
6.657
10
NP PPschalidou et al.
2021
46.35
1.85
43.09
24.43
5
BAE (Chen et al. Ours
9b
24.83
3.211
33.67
52.50
1.42/8
31.87
2.384
36.69
2.149
4.16/8
Reconstruction performance. Chamfer L1 distance (CDI) and the number of parameters (# of params.) are scaled by 10a and 10s, respectively. The average and the predefined maximum numbers of recovered parts or primitives are shown before and after the slash, in the last column.
Table 4:
w/o Lyoo
w/o Ldes
w/o Looe
w/o Luar
w/o Lidd.
w/o CS
w/o CP
w/ all
Label IoU
72.20
73.21
74.27
65.29
70.14
55.67
71.35
80.99
EPE 4
4.362
6.628
9.250
6.676
7.276
8.827
7.219
2.988
Table 5: Ablation study of the losses and the category-common decoders: "w/o CP" and "w/o CS" means disabling the category-commet pose decoder and the shape decoders, respectively.
This is because PPD enables the use of deeper network structure of shape decoders for better ex- pressive power, and shape decoders are robust to unseen part poses owing to the disentangled part pose representation. For the further discussion shape reconstruction, see Appendix G
4.4 ABLATION STUDIES
We evaluate the effect of the proposed losses and the category-common decoders on part segmen- tation and part pose estimation. We disable each loss except Lree and Lug one at time. We also disable the delegory-commoot decoders for pose and shape one by one and only use the cor- responding mustance-dependen decoder(s). The results are shown in Table 5 Enabling all losses and the category-commi decoders performs the best. Particularly, disabling the category-commet shape decoders significantly degrades both label IoU and EPE. This indicates that learning category- common shape prior is essential to perform proper part decomposition and to facilitate part pose learning, which is the core idea of this study. For further ablation studies, see Appendix H
4.5 DEPTH MAP INPUT AND REAL DATA
Because PPD's decoders do not assume complete shape an input, it works with depth map input. Following BSP-Net (Chen et al. (020), we train a new encoder that takes a depth map captured from various viewpoints
partial point cloud and replace the original encoder. We minimize the mean squared eтror between the output latent vectors of the original and the new encoders so that their output are close for the same target shape. The results are shown Table 6 The depth map input performs comparably to the complete point cloud input. We also verify that our model trained on synthetic depth maps reasonably generalizes to real data, as shown in Figure
CONCLUSION
We propose novel unsupervised generative part decomposition method, PPD, for articulated ob- jects considering part kinematics. We show that the proposed method learns the disentangled repre- sentation of the part-wise implicit field the decomposed part shapes and the joint parameters of each part as the part poses, using single-fram shape supervision. We also show that our approach outperforms previous generative part decomposition methods in terms of semantic capability and show comparable part kinematics estimation performance with the supervised baseline. Finally, we confirm that our model also works on the depth map input and generalizes to real data.
F-score
Label IoU f
EPE
Complete Depth
31.42
80.99
2.903
28.99
80.65
3.203
Dopth Prediction
RGB
Depth Prediction
Comparison between the point cloud input types: complete shape and depth map.
Table 6:
Figure 7: Real depth map input. (Left) RBO dataset (Martin-Martin al.) 2018)) and (Right) Articulated Object Dataset (Michel et al. 2015).

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
For the reproducibility, this paper includes the detailed description of our network architecture in Appendix A implementation details on the hyperparameters in Section 3.4 and additional training details in Appendix B including model parameter initialization steps. Not only our models, but we also describe the training details of the baseline models in Appendix B. We also report the detailed steps of the data preparation process for the synthetic datasets in Appendix [ We describe
the further detail on the data split in Appendix C.I and data generation steps as well as publicly available source code that for the initial labeling process used to evaluate unsupervised part segmentation results Appendix
to generate the data in C22 For evaluation, report the steps
D.I
REFERENCES
Ben Abbatematteo, Stefanie Tellex, and George Konidaris. Learning to generalize kinematic models to novel objects. In Proceedings the Conference on Robot Learning (CoRL), pp. 1289-1299, 2020.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey Hinton. Layer normalization. arXiv preprint
arXiv: N.X.:0667.004550 2016.
Suzanna Becker and Geoffrey E Hinton. Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature, 355(6356::611-13, 1992.
I Binford. Visual perception by computer. In Proceedings of the IEEE Conference of Systems and Control, pp. 116-123, 1971.
Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In Proceedings of the European Conference Computer Vision (ECCV), pp. 561-578, 2016.
Aljaž Božič, Pablo Palafox, Michael Zollhöfer, Justus Thies, Angela Dai, and Matthias Nießner. Neural deformation graphs for globally-consistent non-rigid reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1450-1459, 2021.
Ching-Hang Chen, Ambrish Tyagi, Amit Agrawal, Dylan Drover, Stefan Stojanov, and James M Rehg. Unsupervised 3d pose estimation with geometric self-supervision. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 5714-5724, 2019a.
Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. Bae-net: branched autoencoder for shape co-segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 8490-8499, 2019b.
Zhiqin Chen, Andrea Tagliasacchi, and Hao Zhang. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 45-54, 2020.
Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, and Andrea Tagliasacchi. Cvxnet: Learnable convex decomposition. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 31-44, 2020a.
Boyang Deng. John Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi. Nasa neural articulated shape approximation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 612-628, 2020b.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. In Advances in Neural Information Processing Systems (NeurIPS), pp. 5769-5779, 2017.
10

---

Under review as a conference paper at ICLR 2022
Jiahui Huang, He Wang. Tolga Birdal, Minhyuk Sung, Federica Arrigoni, Shi-Min Hu, and Leonidas J Guibas. Multibodysyne: Multi-body segmentation and motion estimation via 3d sean synchronization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 7108-7116 2021.
Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7122-7131, 2018
Yuki Kawana, Yusuke Mukuta, and Tatsuya Harada. Neural star domain as primitive representation. In Advances in Neural Information Processing Systems (NeurIPS), pp. 7875-7886, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69880 2014.
Marian Kleineherg, Matthias Fey, and Frank Weichert. Adversarial generation of continuous implicit shape representations. In Eurographics, pp. 41-44, 2020.
Zoe Kourtzi and Nancy Kanwisher. Activation in human mt/mst by static images with implied motion. Journal of Cognitive Neuroscience, 12(1):48-55, 2000.
Nilesh Kulkarni, Abhinav Gupta, David F Fouhey, and Shubham Tulsiani. Articulation-aware canonical surface mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 452-461, 2020.
Xiaolong Li. He Wang. Li Yi, Leonidas Guibas, A Lynn Abbott, and Shuran Song. Category-level articulated object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3706-3715, 2020.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi person linear model. ACM transactions on graphics (TOG), 34(6):1-16, 2015.
William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construc- tion algorithm. Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. (63,169, 1987.
Roberto Martin-Martin Clemens Eppner, and Oliver Brock. The rbo dataset of articulated objects and interactions. arXiv preprint /806v 06465, 2018.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Oc- the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 4460-4470, 2019.
cupancy networks: Learning 3d reconstruction in function space. In Proceedings the
Frank Michel, Alexander Krull, Eric Brachmann, Michael Ying Yang, Stefan Gumhold, and Carsten Rother. Pose estimation of kinematic chain instances via object coordinate regression. In Pro- ceedings of the British Machine Vision Conference (BMVC), pp. 181.1-181.11, 2015.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2018
Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy J Mitra, and Leonidas J Guibas. Structedit: Learning structural shape variations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8859 8868. 2020.
Despoina Paschalidou, Ali Osman Ulusoy, and Andreas Geiger. Superquadrics revisited: Learning 3d shape parsing beyond cuboids. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 10344-10353, 2019.
Despoina Paschalidou, Luc Gool, and Andreas Geiger. Learning unsupervised hierarchical part decomposition of 3d objects from a single rgb image. In Proceedings IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

---

Under review as a conference paper at ICLR 2022
Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. Neural parts: Learning expressive 3d shape abstractions with invertible neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- $204-3215, 2021.
Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estima- tion in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7753-7762, 2019.
Charles Qi. Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 652-660, 2017.
Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2, arXiv preprint arXiv:I906.0000 190069040000 2019.
Dominic Roberts, Ara Danielyan, Hang Chu, Mani Golparvar-Fard, and David Forsyth. Lsd- structurenet: Modeling levels of structural detail in 3d part hicrarchies. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5836-5845, 2021.
Lawrence G Roberts. Machine perception of three-dimensional solids. PhD thesis, Massachusetts
Institute of Technology, 1963.
Nobu Shirai and Tomoko Imura. Implied motion perception from still image in infancy. Experi- mental Brain Research, 232(10):3779-3387, 2014.
Dong Wook Shu, Sung Woo Park, and Junseok Kwon. 3d point cloud generative adversarial net- work based on tree structured graph convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems (NeurIPS). pp. 7462-7473, 2020.
Alan Slater, Victoria Morison, Carole Town, and David Rose. Movement perception and identity constancy in the new-born baby. British Journal of Developmental Psychology, 3(3):211-220) 1985.
Elizabeth Spelke, Roberta Kestenbaum, Daniel Simons, and Debra Wein. Spatiotemporal con- tinuity, smoothness of motion and object identity in infancy. British Journal of Developmental Psychology, 13(2):113-142, 1995.
Shubham Tulsiani, Hao Su, Leonidas Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2635-2643, 2017.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing in- gredient for fast stylization. arXiv preprint arXiv: 2016
Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao, and Kai Xu. Shape2motion: Joint analysis of motion parts and attributes from 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8876 8884, 2019.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang. Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11097-11107, 2020.
Fei Xu and Susan Carey. Infants' metaphysics: The case of numerical identity. Cognitive psychol- ogy, 30 1996.
Zike Yan and Xuezhi Xiang. Scene flow estimation: A survey. arXiv preprint arXiv: 1612.02900, 2016.
12

---

Under review as a conference paper at ICLR 2022
Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, and Leonidas Guibas. Deep part induction from articulated object pairs. ACM Transactions on Graphics (TOG), 37(6):1-11, 2018.
Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6365-6373, 2017.
Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, and Michael J Black. Three-d safari: Learning to estimate zebra pose, shape, and texture from images" in the wild". In Proceedings of the JEEE/CVF International Conference on Computer Vision (CVPR), pp. 5359-5368, 2019.
A NETWORK ARCHITECTURE
In this section, we explain the detailed architecture of the proposed network. The network architec- tures of the neural networks employed in the proposed method are depicted in Figure The squircle diagram represents tensors, where the first and the second numbers inside the parentheses indicate the channel and the number of points, respectively. The squircle without the parentheses indicates the scalar value. For the split operation, N [[x,7 Y] denotes the split operation the input tensor to N number of sliced tensors with X number of channels with Y points. For the square diagrams with square brackets, the first and the second numbers in the square brackets indicate the input and output channels, respectively. The green square diagrams indicate multiple identical subnetwork architectures. Pe denotes the number of points in the input point cloud to encoder F.. For the other notations, see Section 3
We use the simple PointNet architecture in the author-provided code of (Mescheder et al. 2019) as the encoder E. For the normalization layer in FF.C and 1p.-s we have experimentally found that using instance normalization (Ulyanov et al. 2016) for F and layer normalization (Ba et al., 2016) for FP.s achieves the best performance. For the joint state 8,, we multiply π to {s; 1/4 revolute). For the discriminator D, we use the architecture based on the PointNet (Qi et 2017) implementation in the author-provided code of (Shu et al.) 2019). The weight of each linear layer in our discriminator normalized using spectral normalization (Miyato al. 2018) for stable training.
B TRAINING DETAILS
In this section, we explain the implementation and training details of the proposed models. We train models per category with the same hyperparameter configuration described in Section 3.4 for all categories. For the input, we use the point cloud with 4096 points sampled from the surface of the target shape during the training. Unless otherwise noted, we use the complete shape point cloud. We use a batch size of 18. For the ground implicit field, for each sample in a batch, we use 4096 3D coordinate points and their corresponding indicator values sampled from either 162 or 323 grids, depending on the training stage. This multi-stage training strategy on grids with different resolutions inspired by (Chen al. 2020). We train our network 16 grids in the first training stage. In addition, we set r, = in the first stage. Then, we set r, = rf + r in the second stage. We determine the number of iterations for each stage according to the reconstruction loss and to the visualization of the reconstructed shapes on the validation data. It takes 2 to 3 days to train one model on a single NVIDIA V100 graphics card with 16 GB of GPU memory.
Model parameter initialization. We use a sine function as a nonlinear activation function and the weight initialization strategy proposed in (Sitzmann 2020 in our shape decoders, as follows:
w U
IN
IN
IN
30
(10)
where IN is input channel to linear layer, U is uniform distribution, and is an element of the weight of a linear layer. For a linear layer that takes 3D coordinates as an input, we do not scale the weight w by 30-
13

---

Under review as a conference paper at ICLR 2022
Instane deponddrr
Galagar cocmoar
Insance deponderr
Peesear [ [
Time
Liser Pa
[imes [imee
DI
Figure 8: The architectures of our networks.
Randomily posed
Drawer
Eye- glasses
Oven
Laptop
Washing machine
Training
24
35
30
13
39
Test
7
13
(0)
(1D))
(101)
(101)
Canonically posed
Table 7: Number of samples per category in each data split. Each sample is augmented by transforming its part pose to generate 100 in- stances. Numbers in a parethnesis in the last column indicates the ground truth number of fixed, prismatic and revolute type parts.
Eye-
Laptop
Washing
glasses
machine
Figure 9: Visualization of the canonically posed and randomly posed ground-truth meshes of each category. The colors correspond to the different ground-trute part labels.
B.1 TRAINING OF THE BASELINE MODELS
In this section, we describe the training details of the baseline methods. We use the author-provided implementations.
BSP-Net (Chen et aL. 2020 Because the models in the author-provided codes of the other part decomposition baselines (BAE-Net (Chen al. 201 and NSD (Kawana et al. (020)) trained on 323 grids, we also trained BSP-Net on up to 32 grids, compared to the 64 grids in the orig- inal implementation. For training on the eyeglasses category, we could not successfully train the model even with different random seeds with the provided training script. After several trials, we experimentally found that scaling ground-truth indicator values by four for the first 20,000 iterations produced good initialization of the model. On the basis of this finding, we first pre-trained the model using the scaled ground-trum indicator values for 20,000 iterations for the eyeglasses category; then, we trained the model with the provided training script.
NSD (Kawana et al./ 2020) and Neural Parts (Paschalidou et al., 2021) The model defined in the author- provided code takes RGB image as input, which is more challenging setting for 3D shape reasoning than 3D shape input. We replace the image encoder the original implementation with the same PointNet-based encoder used in our approach for fair comparison.
NPCS (Li et al. 2020). In the experiment described in Section 4.2 we modified the original im- plementation of NPCS to use complete shape point clouds instead of partial point clouds of the depth map as an input with training from scratch, to remove the unnecessary performance degrada- tion caused by pose ambiguity arising from the barely visible articulated part.
C DATA PREPARATION
In this section, we describe data preparation approach.
14

---

Under review as a conference paper at ICLR 2022
C.1 DATA SPLIT
We split our training and test data according to the per-category data split approach introduced in (Li al., 2020). We ensure that the split contains at least six samples per category, except for the laptop category: therefore, the average split ratio is approximately 8:2. For the laptop category, we use 11 samples in the test split make the split ratio comparable with those of the other categories. The number of samples in each split per category is presented in Table
C.2 GROUND-TRUTH IMPLICIT FIELD GENERATION
Following [Mescheder et al. 2019), we generate the ground-truth implicit field by the volumetric fusion of 100 depth images of mesh object. For the mesh object, we sample 100 instances with randomly sampled part poses for each sample. For the pose sampling, we uniformly sample the rotation amount for each joint for the revolute joints. For the revolute joints of all categories except the eyeglasses category, sample the rotation amount between 0 and 135". For the eyeglasses category, we sample between 0 and 90" For prismatic joints of the drawer category, we sample the translation amount between 0 and the maximum amounts of the joints written in the URDF files of each sample in the SAPIEN dataset (Xiang et al. 2020). After we sample a part pose for each instance, articulate the sample in its canonical pose (the rotation amount and translation amount were set to 0 and O, respectively) using the sampled motion amount and ground-truth joint configuration. The canonically posed shape and the randomly posed shape of the same sample shown in Figure(y Finally, we normalize the size and location of the instances following (Mescheder al.) 2019). Specifically, normalize the instances with the maximum extent collected from the instances generated from the same sample.
D PART LABELING PROCEDURE FOR EVALUATION AND PART SEGMENTATION VISUALIZATION
D.I PART LABELING PROCEDURE
In this section, explain the labeling procedure using the ground-truth part labels of the train- ing samples to evaluate the part segmentation performance, following the same procedure used in (Kaaana al. 2020 Deng al. 2020a). First, for each surface point sampled from the ground- truth part mesh of the instance of the training set, we determine the nearest reconstructed part and vote for the ground-trute truth part label of that point. Next, we assign each reconstructed part the part label that has the highest number of votes. Finally, for each surface point sampled from the instance in the test split, we determine the nearest reconstructed part surface and assign the part label of the reconstructed part.
D.2 PART SEGMENTATION VISUALIZATION
To visualize part segmentation, similar to Tulsiani et al. 2017). We first measure the distance be- tween barycentric point ground truth mesh face the surface of each part. Then we assign mesh face the label of the part with the shortest distance to the barycentric point. Lastly, we color each face according to the obtained label.
E SEMANTIC CAPABILITY
E. ADDITIONAL VISUALIZATION OF THE PART SEGMENTATION
We visualize the additional part segmentation results of the proposed approach in Figure 10 Also, we visualize the part segmentation results given various part poses in Figure
E.2 PART SEGMENTATION USING ALL THE TRAINING SAMPLES
In Section 4.1
we show that our method works most efficiently by requiring instances with only a limited variety of poses for the initial annotations. In the experiment discussed in Section 4.1
15

---

Under review as a conference paper at ICLR 2022
Segmentation
Segmentation
Reconstruction
Reconstruction
Drawer
Eyeglasses
GT
Segmentation
Segmentation
Reconstruction
Reconstruction
Laptop
Oven
GT
Segmentation
Reconstruction
Washing machine
Figure 10: Visualization the additional part segmentation results of the proposed approach with various samples. For drawer category, the different between some GT shapes are subtle (c.g.. differ- ence handle shapes), we pick the three samples with distinct shape difference to avoid confusion.
16

---

Under review as a conference paper at ICLR 2022
Input
Ours
Drawer
Input
GT
Ours
Eyeglasses
Input
Ours
Laptop
Input
Ours
Oven
Input
GT
Ours
Washing machine
Figure II: Visualization of the part segmentation results given input shapes with various part poses. Arrows in the figure indicate the ground-truth predicted joint directions.
17

---

Under review as a conference paper at ICLR 2022
Drawer
Eye-
Oven
Laplup
muchine
Washing
ieeaa
(Al))
mean (Canonical)
Cunonical)
Did.
of
Cthen
11.11
74.96
27,500256
NSD
74.63
89.01
NP
(Ph Phiemiiioon of 2021
60.56
64.75
85.
86.22
74.22
74.32
74.31
0.01
74.83
66.25
82.05
86,80
95.0
81.02
80,99
0.04
4,18/8
Table 8: Part segmentation performance. We use all the instances in the training set assign label to each part as well as to the primitives. "Canonical" denotes the mean label IoU only using the canonically posed instances of the training for the label assignment. "Difference" shows the performance drop from the setting that uses all the instances in the training set to the setting that uses only the canonically posed instances. The average and the predefined maximum numbers of recovered parts primitives are shown before and after the slash, in the last column. Our method achieves the same level of the label efficiency with Neural Parts with higher part segmentation performance.
Duwer
alosses
Eye-
Oven
Laptop
Wasting
mochine
meci
2k
5k
13k
z3k
68k
239k
GT
BAE
Mees di
30
6:351
11.11
73.05
25111
50.30
30:17
BSP
26.62
71.14
$5.19
64.41
86.40
66.79
NSD
Tern m mm
34.07
60.05
70.00
20.12
60:90
50 66
Figure 12: Visualization of the train- ing process. The first two rows show the reconstruction results for target shapes having the same part shapes but different part poses. The bot- tom row shows the number of train-
NP Paltinnnt
200
61.10
65.47
77.57
62:75
86.00
20.71
D0s
74.75
66.18
82.00
86.81
95.15
$6.99
Table 9: Part segmentation performance in label IoU with the aligned number of parts for all methods (N = 8). The starred numbers indicate the failure of part decomposition and that only one recovered part represents the entire shape.
ing steps.
use canonically posed shapes, visualized in Figure in the training set for the initial annotations. This section reports the evaluation setting where annotations of all training instances are available for the initial annotation, which favorable setting for the baselines. However, the annotation cost can be much higher in reality than in the previous setting. The results are shown in Table S Even under this setting favorable for the previous works, our method performs comparably with part decomposition method BSP-Net (Chen et al. 2020) using 256 primitives. It
the state-of-the-ee is not surprising that using many primitives achieves fewer part segmentation errors because, even when one primitive is inconsistently assigned to the ground-truth part, the impact on the label loU is smaller. This is because smaller portion of the evaluation points becomes erroncous compared with the model using fewer parts primitives. Note that our research focuses on representing ground-truth articulated parts with consistently the same reconstructed parts by considering the part kinematics, unlike BSP-Net and the other baselines, which can assign different sets of primitives to the same articulated parts without considering the underlying part pose. To show the effectiveness of considering the part kinematics, we show the performance drop from using all training instances to using only the canonically posed instances in the table under the heading "Difference." We can see that our approach has the second best drop with the comparable number with Neural Parts (Paschalidou et al. 202I). yet higher part parsing performance. This shows that considering the part kinematics contributes to label efficiency by reducing the necessary initial annotation to perform well the unsupervised part segmentation of articulated objects.
E.3 PART SEGMENTATION WITH THE ALIGNED NUMBER OF PARTS
In this experiment we align the predefined maximum number of the parts of the baselines to same as ours (N = 8) to make the result more comparable. We change the predefined maximum number of primitives from 256 to 8 for BSP-Net (Chen et al. 2222), from to 8 for NSD (Kawana et al. 2020). and to 8 for Neural Parts (Paschalidou et al. (021). Our method outperforms all the baselines on average with larger margin compared to the result in Table of the main paper. The results shown in Table The smaller number of parts worsens NSD and BSP-Net's performance, and the opposite applies Neural Parts.
18

---

Under review as a conference paper at ICLR 2022
Pivot point qi
Pivot point qt
min q: x||
min ||q: - x|| xES
min 11 x|| XCS
S, {x R (0,(8xx)) 0.53
Sge (x E R 1x(x) = 1}
Sij fx R (8 [x 0.5,j 6 AT/ /1)
Figure 13: Illusrationn Lioe in 2D.
joint accuracy excopt crawer)
jenn direction socuracy (AB)
joink distance accuracy
100.0
Acouncy 10 50.0
90.0
ACQuacy
60.0
00.0
60.0
6.30 Thowooadd
Threwhold (Tlg))
Threshold (Deg)
(Drowar)
100.0
(Drower)
320n0 (80
80.0
0.05
0.10
0.30
10.0
20.0
30.0
40.0
Threshold
Threshoid (Deg)
point state accuracy (Eyoglasses)
jentt diroction accuracy (Eyoglasees
Jonn anls distance acouracy (Eyoglaszes)
100.0
acuncy 150
Accuncy
80.0
Accunccy
Ours NPC5
60.0
Our
NPCS
Ours
20.0
30.0
20.0
30.0
40.0
10.0
20.0
30,0 (Deg)
60.0
0.07
0.30
0.35
0.20
(Deg)
joitt state acouracy (Lpptpp)
Joint direction accurocy (apptop)
John ani distance accuracy (Laptop)
Acunsyy (50
39.5
Accorscy (98)
00.0
39.0
80.0
Ours NPCS
98.5
Ours
NPCS
Ours
70.0
30.0
30.0
98.0
10.0
20.0
36.0
40.0
0.05
0.30
0.15
0.20
Threshold
100.0
pint state accurscy (Oven)
100.0
joint direction accurase (Gven)
100.0
jant distance accuracy (Dwin
Accuncy 688
00.0
Accurcc
50.0
80.0
70.0
0.0
Threshold (Deg)
Threshold (Deg)
Threshold
discctio ascunacy
acsuracy (Wihhing machine)
100.0
75.0
Accurnecyc.
50.0
25.0
80.0
25.0
0.0
25.0
16.0
20.0
35.0
46.0
0.30 Threshold
0.20
Threshold (Ocg)
Thrishold (Deg)
Figure 14: Joint parameter estimation performance.
19

---

Under review as a conference paper at ICLR 2022
F-score (mean)
Drawer
Eye- glaners
Oven
Laptop
Washing mathine
mese
100
of assigned parts
1.D
1.0
1.0
L.D
1.0
1.0
Part type
F-score (%)
80
39.50
83.25
100.0
accuracy
Ours
60
NSD
Table 10: Evaluation results of the number of assigned reconstructed parts to the ground-truth parts and the part type accuracy. "# of assigned parts" shows the number of reconstructed parts assigned to the ground-truth parts, and "Part type accuracy" shows the percentage of part kinematic type matches between the ground-trute and the assigned reconstructed parts.
BAE
40
BSP
NP
1
4
10
Threshold (%)
Figure 15: Plot of F-score on average with various thresholds.
F ADDITIONAL PART POSE EVALUATION
Because we train our model in an unsupervised fashion, through the labeling process described in Appendix D.I] the part kinematic types of the ground-truth and the assigned reconstructed part do not necessarily match. Moreover, multiple reconstructed parts may be assigned to one ground- truth part. Therefore, we choose EPE as the evaluation metric for part pose estimation due to its kinematic type agnostic property and calculation based on point correspondence between prediction and ground-truth, rather than part-level correspondence. In this section, as an additional part pose evaluation, we evaluate the accuracy of joint parameter estimation for "revolute" and "prismatic" parts. To avoid the problem of part pose evaluation in unsupervised learning described above, we evaluate the accuracy of joint parameter estimation by considering the prediction is correct when the following three conditions are all satisfied. (1) One reconstructed part is assigned to one ground-truth dynamic part. (2) The part kinematic type is the same between the ground-truth and the assigned reconstructed part. (3) The error of the joint parameters against the ground-truth is less than the error threshold. This evaluation method is more challenging than EPE because of the influence of (1) and (2) above, besides the prediction error of the joint parameters. We evaluate joint state accuracy and joint direction accuracy. Only for the revolute part,
we also evaluate joint axis distance accuracy, defined as the line to line distance between the ground- truth and the predicted line segments consisting of the pivot point and the joint direction. Figure [4] shows the evaluation results with varying error thresholds. We show the results of NPCS only as reference; NPCS a supervised model and assumes that the part segmentation available during training. and the part kinematic types are also known. In contrast, segmentation and part kinematic type in an unsupervised fashion. Since NPCS does not estimate the pivot point, we only show the results of our method for joint axis distance accuracy. As for the joint state, we see reasonable accuracy of 70.80% for revolute parts on average when the threshold less than 10 degrees and 79.43% when the threshold is 15 degrees. For the "prismatic" part of the drawer, our method outperforms the NPCS when the threshold is less than 0.1. For joint direction estimation, in three out of five categories (eyeglasses, laptop, and oven), our method is comparable or outperforming NPCS. In Table [0] we also show the number of reconstructed parts assigned to the ground truth parts and the percentage of part kinematic type matches between the ground-truth and the assigned reconstructed parts. In all categories, the model correctly assigns one part. Moreover, even without part type supervision, our model successfully predicts correct part types with high accuracy of 91.46% Improving the unsupervised learning of joint parameters under shape supervision an interesting research direction.
method learns both part
G ADDITIONAL DISCUSSION ON SHAPE RECONSTRUCTION
Due to the space constraint the main paper, we show category-wise shape reconstuction result in Table I1] Although it is not our focus to beat the state-of-the-ar methods (Chen al.) 2020 Kawana et al. 2020; Paschalidou et al.) T02T) in shape reconstruction accuracy, there are several possible reasons for the performance gap in shape reconstruction. The first reason is that our method relies on the challenging part pose estimation for shape reconstruction. Although our method estimates faithful part pose as evaluated in Section 4.2 and Appendix F, even the small pose error affects shape
20

---

Under review as a conference paper at ICLR 2022
Eye-
Washing machine
Drawer
glasses
Oven
Laplop
meaan
Params
parts
RSP Chen 2020
40.84
51.21
44.81
65.92
56.46
$1.85
443.3
7,50//56
NSD Kawana Ar20
48.63
50.7
37.56
83.58
42.5
52.39
6.657
10
F-gcore
NP [P Pachalidos
2121
43.50
52.68
31.52
72.21
31.84
46.35
24.43
BAE ( Ien si.
2019
24.28
26.34
18.32
31.91
23.12
24.33
52.52
L,2/8
30.97
35.45
28.29
40.19
24.45
31.87
2.149
4.16/8
hem
2020
27,50/256
NSD Kawann el 3
4024
1.594
1.642
2.819
0.649
2.128
1.766
6.657
10
NP PPachallddz
2021
1.958
1.528
2,533
0.823
2.399
1.848
24.43
BAL Chen a.. 2719
2.360
1918
4.314
1.867
3.595
3:211
52.32
L42/8
Duss
1.998
2.375
3.135
1.481
2.930
2.384
2.149
4.16/8
Chen al.
2020
49:07
39.84
56.46
64.73
64.99
55.02
443.3
27. 27,30/256
NSD Knwwan or 2020
52.57
33.58
46.96
71.97
51.50
51.32
6.657
10
loU'
Pacchalideu
2121
35.31
38.21
BAL ben el
20196
40.71
22.12
20.83
45,30
29.41
33.67
52.52
L4278
43.19
4.16/8
Table 11: Reconstruction performance. Chamfer LI and the number of parameters (# of params.) scaled by 102 and 10 respectively. The average and the predefined maximum numbers of recovered parts or primitives are shown before and after the slash, in the last column.
Label IoU
EPE !
w/o Lyee
51.78
11.884
Drawer
Eye- glasses
Oven Laptop
machine
Washing
mean
w/o VQ
72.78
10.772
used
w/ all
80.99
2.988
3
4
4
3
3.4
Table 12: w/o Tfee indicates dis-
Table 13: The number of used constant vectors for the
abling the second term of Equation 2 and w/o indicates disabling use of multiple constant vectors.
category-commin pivot points l E Ar). The maxi- mum number is Ngt
reconstruction scores, especially for the dynamic part type. In Figure 10 the second sample of the eyeglasses category and the first sample of the oven category good examples. Moreover, method regularizes instance-dependent shape decoder's output as described regularization results in less accurate shape reconstruction constrained by the shape prior than the ttate-of-the-an primitive-based shape representations. In Figure I0 for example, in the eyeglasses category, our model captures the global target shape variation yet struggles to recover the smaller details such as the front parts. To confirm how much reconstruction eттог those limitations could add to the performance gap with the state-of-the-art methods, we also evaluate F-score with various thresholds ranging from 0.01 to 0.1. Those thresholds can be intepreted as 1% to 10% distance error threshold for shapes whose side length normalized 1. The result is shown in Figure [5] On the reasonably moderate thresholds (4% to %), our method reaches to the similar performance to the structured reconstruction methods. Improving reconstruction accuracy can be a important future work.
Section 3.2 This
H ADDITIONAL ABLATION STUDY
As additional ablation study, we also verify the effect of the second term of Equatioon]aan and the use of multiple constant vectors to model multi-modal category-common biases of joint configuration by vector quantization (Razavi et al. 2019) discussed in Section 3.1 The results is shown in Table 12 Disabling the second term of Equation significanty drops the part segmentation performance and part pose estimation performance. Disabling the multiple constant vectors particularly affects the part pose estimation performance. Although we have found the category-common joint directions {r. l E AP) tend be encoded in the single constant vector, have also found category-comme pivot points {qg E Ar} effectively utilizes the multiple constant vectors. We show the number of used consant vectors of the model reported in Table in the main paper in [33] We can see that multiple constant latent vectors used to decode the category-commet pivot points.
21


Reviews:
[{"review_id": "xfG8ikvATpO", "reviewer": "Reviewer_YkRG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n+ It is an unsupervised approach for parts decomposition which does not require parts labelling. \n+ It can handle objects with varying number of parts.\n+ It models the part poses of different kinematic types which can better model the articulated objects for unsupervised part decomposition.\n\nCons:\n1)\tIt is not clear how to avoid the cases where multiple implicit fields are used to model the same part. Will the spread of the pivot points discourage the overlapping of the parts?\n2)\tIt would be good to provide details about how to obtain the vector c^p.\n3)\tPlease define the semantic meaning. It is not clear how each part is guaranteed to have semantic meaning.\n4)\tIs the parameterisation of the part pose correlated to the part semantics? In the implementation section, the number of different joint types are predefined. Will this implicitly indicate the parts structure? What is the influence of this predefined numbers on the segmentation results?\n5)\tWhat are the overlappings between different parts?\n6)\tIt is not clear why not compare with Neural parts (Despoina et al, CVPR 2021) which is the most recent unsupervised parts segmentation method.\n7)\tThe parts visualisation in Figure 4 is impressive. However, it is not clear how the drawers are segmented including the interior parts that are not visible from the surface defined by the point cloud.\n8)\tPlease highlight the best performance in the Table 4. Based on the listed numbers, the proposed method does not achieve the best performance in terms F-score and Chamfer distance.\n9)\tIt is not clear what the redundant parts learn if the parts number of the objects is smaller than the predefined maximum number.\n10)\tIt would be interesting to visualise the category-specific shapes.\n11)\tMinors: The writing of the paper should be further polished. …poses can effectively addresses -> address; \nPlease explain semantic capability when it firstly appears in the text (above related works)\nPlease be consistent about the definition of {F_i^{s,c}}_{i=1}^N and {F_i^{s, z}} within the method section (see confusing definitions in the paragraphs above and below Sec. 3.1) Is {F_i^{s,c}}_{i=1}^N the category-common decoder or instance-dependent shape decoder?\n", "summary_of_the_paper": "This paper addresses the problem of unsupervised parts decomposition for articulated objects with kinematic structures. In particular, the proposed approach disentangles the shape and pose of each part separately. Instead of adopting the primitive-based shape representation, each part is represented as an implicit field and disentangled part poses of three kinematic types. A category-specific shape/pose prior and instance-dependent shape/pose details are learned separately. The proposed approach is compared with existing works on benchmark datasets.", "summary_of_the_review": "Overall, the proposed unsupervised parts segmentation approach from disentangled shape and pose is novel. However, the paper is not well-written. Details are not clear to the reviewer. The results are not quite promising as demonstrated in Table 4. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "vxKnUpP66Sc", "reviewer": "Reviewer_4Pds", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "STRENGTHS\n- Well-written paper, detailed introduction, and motivation of the work.  \n- Good description of the method, with sufficient details to allow re-implementation and deep understanding of the implementation.\n- Good evaluation, where the different tasks enabled by the method are evaluated: semantic segmentation, par pos estimation, and reconstruction. Also ablation studies are provided.\n- First method to do unsupervised decomposition of articulated objects.\n\nWEAKNESSES\n- I think comparing to BAE and BSP can be a little bit misleading since these methods do not assume articulated objects. Working under the 'articulation' assumption allows the authors to frame their loss, and it is understandable that testing on articulated objects works better. A more fair (or additional) comparison would be to use this method on non-articulated objects and compare it to BAE and BSP.\n- The symbols used for encoders and decoders are too cryptic and hard to parse while reading the paper (e.g. $F^{p,z}$, $F^{p,c}$, $F^{s,z}$, etc.). I'd encourage the authors to rename those to more meaningful names to ease reading.", "summary_of_the_paper": "This submission proposes an unsupervised method to segment point clouds of objects that are articulated. In contrast to previous work with similar tasks, the learning is done in an unsupervised manner, outputting a segmented point cloud represented as a combination of implicit fields that can be rigidly articulated. The paper is well presented, technically sound, and correctly evaluated and compared to works with similar goals.", "summary_of_the_review": "I think this is a nice written paper with a good technical contribution. The paper is well written and the results look convincing. Due to the specific scope of the paper (i.e., articulated objects) and the lack of works with this specific goal, comparisons to SOTA methods are somewhat difficult to parse. Nonetheless, seems to propose a step forward in the scenario that is designed for. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "sjgcW_R4uZ", "reviewer": "Reviewer_bBs4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths:\nThe proposed method attacks the very challenging problem of capturing the shape of an articulated object, together with its variations within the object category, its decomposition into parts in a parsimonious way, and the estimation of each part's pose in terms of a joint state and, in the case of revolute joints, a pivot point. \n\nAlthough the overall architecture is somewhat complicated with numerous loss terms involved, the proposed method achieves improved performance with respect to other unsupervised generative part decomposition methods on standard metrics considered for the task. In terms of part pose estimation, it achieves comparable performance to the NPCS which assumes part segmentation supervision, even though the proposed method is unsupervised. Additionally, the part parsing produced is more parsimonious and intuitive qualitatively, in terms of interpretability. \n\nThe experimental evaluation, although compact, is quite comprehensive covering many aspects of the method. It considers also a different input modality (depth maps) as well as the application of the method on real data. Also, the authors have made significant effort to assure that comparison with other baselines is fair. \n\nRegarding reproducibility, all the implementation details are provided, making it possible to reproduce the results. It would also be important though to make the source code available.\n\n# Weaknesses:\nThere are few weaknesses I can identify. One is that the notation is quite heavy, involving numerous symbols and many superscipts/subscripts. Admittedly, given the overall complexity of the task and the method proposed, it might be difficult to make the notation lighter. One thing that might help, would be to include full words in the subscripts of the loss terms, as their number is high and it is not easy to remember what each of them tries to achieve. Nevertheless, given the compactness of the Methods section, the paper manages to provide a quite clear explanation of the method.\n\nA minor issue, in my view, is that the fact that a discriminator is used is stated quite late in the paper (Section 3.2, pg. 6). I would prefer to know earlier (possible from the introduction), that the method is based on an adversarial learning setting.\n\nAnother issue of the overall presentation regards the hyperparameters. Some of them, like e_i and y_i, provide crucial information to make learning possible and seem to depend to some extent on the structure of the particular objects, in general, and the way they are represented in the dataset, specifically. I would prefer if these hyperparameters were presented and discussed in a more straightforward way, rather than been less evident within the text.\n\n## Minor Comments\n- Introduction: \"have mainly focus\"\n- Figure 1: Left and Right labels should be swapped\n- Notation: \\mathcal{X} is not properly defined, this makes difficult to interpret the domain of \\hat{O}_i (R^3\\times \\mathcal{X})\n- The phrase: \"Instead of using a single constant vector, ... biases\" (before eq.4 on pg.5) is not very clear\n- Tables: please indicate which metrics are higher-better and which are lower-better\n- Appendix B: Wrong citation for NPCS ", "summary_of_the_paper": "The paper presents a method for unsupervised generative part decomposition based on part kinematics. By employing a common encoder, operating on the input point cloud, the method learns a non-primitive-based implicit field for each part using a set of category and instance specific shape decoders, and the joint parameters of each part using a set of category and instance specific pose decoders. ", "summary_of_the_review": "Overall, I think that the paper is quite strong and the method proposed contributes some quite interesting ideas for effectively addressing the challenging problem of unsupervised pose-aware generative part decomposition. The experimental evaluation is comprehensive and follows correct practices.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VoOYKXgEfWJ", "reviewer": "Reviewer_NczN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths:\n------------\n1. I really like the direction of having generative part-based models. I think this topic has not been adequately explored and while this paper does not provide enough experimental evidence on generative tasks, I consider it an important step towards developing generative models for part-based methods.\n2. The paper seems technically sound and the proposed network architecture is novel.\n3. I appreciate the authors's efforts in providing extensive experiments both in the main paper and in the appendix.\n\n### Weaknesses:\n-------------\n\n1. My major concern for the paper is related to its clarity. While I overall like the proposed model, I found it quite hard to follow the paper. First of all, the goal of the proposed method is not crystal clear. In some parts of the text it is mentioned that the main focus of this work is developing a generative part-based model (e.g. 3rd paragraph in the Related Work Section), whereas elsewhere it is stated that the focus of this work is on learning shape abstractions (e.g. first sentence of Method Section). I do not think that the authors sufficiently demonstrate that the proposed method can be used for generative tasks, since they only show some interpolation results in Figure 5. However, this doesn't diminish the contributions of this work. Therefore, for future version of the paper, I would recommend that the authors try to ease their claims regarding their model being able to operate on generative tasks. Secondly, I think that the Method Section should be restructured in order to improve the paper's readability. Please see my comments below for more details. Finally, closely related to my first point, also the experiment section does not properly reflect the contributions of this work. Namely, just by looking Table 4, a reader would probably think that the proposed method is not the best choice if we are interested in accurate reconstructions, since it is consistently worse than all baselines wrt Chamfer loss for all object categories and also performs worse wrt F-score in the 4 out of 5 object categories.\n2. My second concern is related to the dataset that the authors decided to use. I think the current dataset is quite simple. Since the authors are interested in articulated objects, it might be useful to also consider the D-FAUST dataset that contains humans performing various actions. I believe that the claims of the paper would be significantly stronger if the authors were showcasing that their model can also operate in more challenging setups.\n3. While I appreciate the author's efforts in providing extensive experiments both in the main paper and in the supplementary. For the final version, I would like to see additional qualitative results on more object instances for the various categories. From the current experiments, it looks to me that the same objects instances are repeatedly used. Showing additional examples, would only make the claims of the paper stronger.\n\n### Questions / Detailed Comments:\n-------------------------------\n\n1. For the experiment in Table 2, what is the number of ground-truth parts for each object category (on average)? Are these numbers reported in Table 7 in the Appendix? I think it would be better to fix the number of parts for all baselines (similar to Fig 3.) so that the results are comparable because with the current comparison it is unclear whether the superiority of the proposed model stems from the fact that it is using fewer components, which result in more coarse abstractions.\n2. For the experiment in Table 4, can the authors clarify how is the F-score computed? In addition, how many points are used for computing the Chamfer-L1 loss? Is there any reason why the authors are not reporting the mean-IoU which is also commonly used for evaluating the reconstruction accuracy of primitive-based representations? I believe that including an additional comparison wrt the IoU would make this experiment more complete. Finally, it might also be useful to compare the proposed method with the more recent work of [2] since the authors demonstrate that they can capture complex geometries with few components.\n3. For the experiment in Table 4, can the authors please clarify how the authors compute the number of learnable parameters for each method? Instead, I think it would be better, to report the number of primitives used by each method, as in Table 2.\n4. Can the authors describe their intuition for the second term of Eq. 2. Is the second term really necessary? Unless I am misunderstanding something, I would expect that only using the first term of Eq. 2, which is the standard reconstruction loss for the majority of part-based models that rely on implicit surfaces, should yield good results? Can the authors elaborate on this and maybe also add a small quantitative comparison with and without the second term of Eq 2?\n5. I am wondering whether the use of the VQ-VAE is really necessary. While I understand the authors's argumentation regarding its use allowing to capture discrete, multi-modal category common biases, I believe it would be beneficial for the paper to also ablate its use. Therefore, I recommend that the authors include this experiment for the final version of their paper.\n6. In page 4, in Section 3.1, the authors state that their model first tries to reconstruct the target shape with a single part and as the training progresses it focuses on the remaining parts. This is also nicely shown in Figure 11 in the Appendix. Can the authors provide some intuition regarding why this is the case? For the majority of part-based methods, the models tend to distribute all parts on the target shape and then as the training progresses, each part focuses on a specific region of the target object. Why is this different for the proposed method? Moreover, if the model manages to capture the geometry of the target object using a single part, what is the incentive for utilizing multiple components in later training stages? I personally, find it quite hard to explain why this is the case.\n7. Since the proposed method performs worse than all baselines wrt Chamfer loss and F-score (see Table 4), I believe the authors should try to describe why this is the case. Intuitively, I would have expected the proposed method to perform comparable to BSP-Net, however BSP-Net seems to be significantly better. My intuition is that for BSP-Net the authors are using significantly more primitives, thus the discrepancy. However, it would be nice to also elaborate more on this.\n\n### Suggestions:\n--------------\n\n1. In page 3, in Section 3, in the second paragraph, where the authors briefly describe the components of their architecture, I think it would be helpful to add a sentence briefly summarizing the purpose of each component intuitively and at a higher level. Currently, from reading this paragraph it is hard to say what is the purpose of the having a category-common shape decoder and an instance-dependent shape decoder. Simply adding a sentence elaborating on the purpose of each component would significantly improve the paper's clarity. Similarly, I would really appreciate it if the caption of Figure 2, was more detailed. Specifically, it would be nice to provide the higher level concept of each component in order to allow the reader to better comprehend how the model works.\n2. I think it would be great to move Figure 12 from the appendix to Section 3.2 in the main paper. The notation in this chapter is quite heavy and adding a figure that shows each variable would definitely be useful for the reader.\n3. In the caption of Table 2, it would be better to mention what is metric that is reported in the corresponding table. In addition, in the table caption, instead of mentioning generated parts, I think it is more accurate to mention recovered parts.\n4. In Section 4.1, the authors state \"We can also see that our part shapes are more semantic and interpretable than the previous works.\". I am not sure whether this is 100% true, since the final decomposition in particular for the case of NSD seems to be quite comparable with the proposed method.\n5. I think that the Method Section can be restructured. I recommend to first describe the part pose parametrization (Section 3.2), the introduce the part shape representation through the implicit functions (Section 3.1) and finally add a separate section that presents and describes the loss terms of Eq. 2,3,4,5,6,7,8,9.\n6. Throughout the text, the authors say that they have \"single-frame shape supervision\". Maybe instead say that the supervision comes in the form of a watertight mesh parametrized as a set of occupancy pairs indicating whether a point lies inside or outside the target object.\n\n[1] Dynamic {FAUST}: Registering Human Bodies in Motion, Bogo et al. CVPR 2017\n\n[2] Neural parts: Learning expressive 3d shape abstractions with invertible neural networks., Paschalidou et. al. CVPR 2021\n\n### Minor Comments / Typos:\n-------------------------\n- In page 2, \"To address the problems associated with (1) kinematically learning ... can effectively addresses these problems.\": the authors should state how the above concerns are addressed with the current model.\n- In page 3, \"In contrast, our approach focuses on the man-made articulated object of the general category with various kinematic structures.\": instead I would rephrase to \"In contrast, our approach focuses on man-made articulated objects of a category with various kinematic structures.\"\n- In page 3, \"PPD employs an autoencoder architecture, and trained ...\": instead it should be \"is trained\"\n- In page 3, \"Given a single-frame point cloud I ...\": maybe remove \"single-frame\", I personally find it a bit confusing\n- In page 5, \"Through the observation, we assume ...\": It is not clear to which observation the authors refer to.", "summary_of_the_paper": "The proposed paper introduces a pose-aware part-based method for articulated objects. The proposed model takes as input a pointcloud and predicts a set of part poses as implicit functions. The network architecture comprises an encoder that maps the input pointcloud to three latent vectors, a category-common decoder and an instance pose decoder that take the two latent vectors and regress the part pose parameters and a set of shape decoders that are used for generating the part-wise implicit fields. To the best of my knowledge, this architecture is novel and the authors demonstrate that their model yields better segmentation results than various baselines on the Motion dataset. Moreover, the authors evaluate their model wrt the reconstruction accuracy of the final prediction. From the provided analysis it seems to perform worse than existing baselines. Finally, while the authors claim that their model can be used on various generative tasks, they only provide some interpolation results. To summarize, I think this paper is an important step towards pose-aware primitive-based representations, however some parts of the text require further clarifications.", "summary_of_the_review": "While I like the paper, some aspects are unclear to me and I expect the authors to clarify them during rebuttal. Regarding the paper's clarity, I am confident that the paper can be significantly improved if the authors spend some time restructuring and rewriting parts of the text. Finally, I believe that showing additional qualitative results and ideally also providing results on a more challenging dataset would make the claims of the paper significantly stronger. Therefore, I think that the paper in its current state is below the threshold of acceptance. However, if the authors address my concerns I am happy to increase my initial score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission received split reviews: two reviewers recommended accepts, and the other two rejects. The AC went through the reviews, responses, and discussions carefully. The AC appreciates the authors' effort during the response period and agreed that the revision has addressed some of the concerns of the reviewers. However, a few key issues are not fully addressed. This includes results on additional, more complex object categories; discussion on why the performance of the proposed method is not even as comparable as BSP-Net (Table 4); and others.  Further, while the authors have significantly refactored the paper to address the concern on presentation and clarity, the changes are too major for the reviewers to review during the response period (the reviewers are expected to check minor updates, but not review a new paper during the response period).  

Considering all pros and cons, the AC recommends rejection. The authors are encouraged to revise the paper for the next venue.
Decision: Reject